Develop Reference

Why does IQueryable.Select use the same reference for each iteration

I've got an IQueryable on which I want to execute a Select. In that select I create a new instance of an object and run a function which copies the values of the object form the IQueryable(b) to to newly created object(new DTO) and then returns this instance.
IQueryable.Select:
businessLayer.GetAll().Select( b => new DTO().InitInhertedProperties(b)).ToList();
Function in DTO:
public DTO InitInhertedProperties(Base baseInstance)
{
return Utilities.InitInhertedProperties(this, baseInstance);
}
Function for Copying:
public static T InitInhertedProperties<T,K>(T instance, K baseClassInstance) where T : K
{
foreach (PropertyInfo propertyInfo in baseClassInstance.GetType().GetProperties())
{
object value = propertyInfo.GetValue(baseClassInstance, null);
if (null != value) propertyInfo.SetValue(instance, value, null);
}
return instance;
}
The first time the InitInhertedProperties method gets called instance is an empty object, baseClassInstance has the values that it should have:
The result of the first iteration looks like this:
As you see: Everything worked out like it should be on the first iteration. Now the second iteration.
The second time the InitInhertedProperties method gets called insatnce isn't a new instance, but the one of the first iteration. The baseClassInstance is exactly what it should be:
The result of the second iteration looks like this:
The resulting list looks like this:
This only happens when using IQueryable.Select. When using List.Select the result looks just like expected.
That means doing this fixed the issue. But it's just a work around not the solution.
businessLayer.GetAll().ToList().Select( b => new DTO().InitInhertedProperties(b)).ToList();

When you working with IQueryable you are bound to Expressions. Entity Framework will inspect each expression that you put inside Select, OrderBy and other methods and try to translate it to SQL. So you can't call arbitrary methods inside your lambda, only known by EF
If you want to do something, that does not have a direct support from SQL engine you can call AsEnumerable:
businessLayer.GetAll().AsEnumerable().Select( ...
(Please note, that AsEnumerable is better than ToList because it keep laziness)
Another option that may (or may not, depending on Query Provider version) work is to build expression manually:
public static Expression<Func<TEntity, TDto>> InitInhertedProperties<TEntity, TDto>() where TDto : TEntity
{
var entity = Expression.Parameter(typeof(BusinessObject), "b");
var newDto = Expression.New(typeof(Dto).GetConstructors().First());
var body = Expression.MemberInit(newDto,
typeof(TDto).GetProperties()
.Select(p => Expression.Bind(p, Expression.Property(entity, p.Name)))
);
return Expression.Lambda<Func<TEntity, TDto>>(body, entity);
}
Usage:
var myExp = InitInhertedProperties<BusinessObject, Dto>();
var result = businessLayer.GetAll().Select(myExp).ToList();

Get List of Objects Added to Entity Framework 6 Include List

Background
I have created object graphs in Entity Framework where any given object A will have a table Ac that tracks changes for it. These objects may also connect to each other, such as A being 1-many to B. Here is an example graph:
A -> Ac
/ \
Bc <- B \
/ \
Cc <- C D -> Dc
I want to be able to load an object and specific connected objects at a point in time by using the change tables to pull those records and apply them. Ideally, I'd like to be able to either use or mimic the .Include function from Entity Framework.
The Issue
Pulling out which objects are already included in an IQueryable is not as easy as I guessed it would be. Looking at an IQueryable<T> with a child object of T Include()-ed, I can see that these relationships are stored in some sort of Span object within an Arguments property - but these are both internal classes and trying to retrieve this information has a lot of steps.
Here is what I have so far:
public static void LoadVersion<T>( this IQueryable<T> query, DateTime targetDateTime )
{
//grab the value of the "Arguments" property on query.Expression
//this has to be done through reflection because "Arguments" is not accessible otherwise
PropertyInfo argumentsPropertyInfo = query.Expression.GetType().GetProperties().FirstOrDefault( x => x.Name == "Arguments" );
dynamic argumentsPropertyValue = argumentsPropertyInfo.GetValue( query.Expression );
for (int i = 0; i < argumentsPropertyValue.Count; i++)
{
//This gets me a System.Data.Entity.Core.Objects.Span, but that class is internal
//In the watch, I can see span -> SpanList[0].Navigations[0] gives me the name of the class in the .Include()
// This is the value I need
dynamic span = argumentsPropertyValue[i].Value;
//So if I try to pull it out using the same reflection trick as before, I get
// a dynamic {System.Reflection.PropertyInfo[0]} (not a list, as you would normally expect),
// and accessing those values & methods makes the debugger exit without an exception
dynamic spanPropertyInfo = argumentsPropertyValue[i].Value.GetType().GetProperties();
//this makes the debugger exit without an exception
dynamic spanPropertyValue = spanPropertyInfo[0].GetValue(span);
//this also makes the debugger exit without an exception (with the above line commented out, of course)
dynamic spanPropertyValue2 = spanPropertyInfo.GetValue( span );
}
}
Based on how difficult it is for me to find what is Included in a Query, I can't help but think that I am doing this entirely the wrong way. Digging through some of the Entity Framework 6.1.3 source code hasn't shed much light on this.
Edit
I've been playing around with the code provided by Alex Derck, but I realized I still need a few pieces to make this work the way I want.
Here is the version of VisitMethodCall I implemented:
protected override Expression VisitMethodCall( MethodCallExpression node )
{
if (node.Method.Name != "Include" && node.Method.Name != "IncludeSpan") return base.VisitMethodCall(node);
try
{
string includedObjectName = (string) node.Arguments.First().GetPrivatePropertyValue( "Value" );
if (includedObjectName != null)
{
_includes.Add(includedObjectName);
}
}
catch (Exception e ){ }
return base.VisitMethodCall( node );
}
I'm able to construct a query with includes and get the names of the objects I included using the IncludeVisitor, but the main goal to use these was to be able to find the related tables and add them to the include.
So when I have the equivalent of this:
var query = ctx.Persons.Include(p => p.Parents).Include(p => p.Children);
// includes[0] = "Parents"
// includes[1] = "Children"
var includes = IncludeVisitor.GetIncludes(query.Expression);
I am successfully grabbing the includes, and I can then find the related tables (Parents -> ParentsChanges, Children -> ChildrenChanges), but I'm not 100% sure how to add these back to the include.
The main problem here is when it's a nested statement:
context.A.Include(x => x.B).Include(x => x.C).Include(x => x.B.Select(y => y.D))
I can successfully traverse that whole graph and get the names of A, B, C, and D, but I need to be able to add a statement like this back to the include:
[...].Include(x => x.B.Select(y => y.D.Select(z => z.DChanges)))
I can find DChanges just fine, but I don't know how to build that include back up because I don't know how many steps are between DChanges and the original item (A).

After looking a bit in the source code of Entity Framework I noticed the includes are not part of the Expression, but rather part of the IQueryable. If you think about it, it's pretty obvious it should be that way. Expressions can't actually execute code themselves, they are translated by a provider (which is also part of the IQueryable), and not all providers should know how to translate an Include method. In the source code you can see the IQueryable.Include method calls the following small method:
public ObjectQuery<T> Include(string path)
{
Check.NotEmpty(path, "path");
return new ObjectQuery<T>(QueryState.Include(this, path));
}
The query (casted to an ObjectQuery) simply gets returned and only it's internal QueryState gets changed, nothing at all happens to the expression. In the debugger you can see the EntitySets that will be included if you look in the IQueryable, but I haven't been able to put them into a list (_cachedPlan is always null when I try to access it through reflection).
I think after seeing this, the thing you're trying to do is not possible, so I would keep a static list of strings in my dbContext and implement a custom Include extension method:
public partial class TestDB
{
public static ICollection<Expression> Includes { get; set; } = new List<Expression>();
public TestDB() : base()
{
Includes = new List<Expression>();
}
...
}
public static class EntityExtensions
{
public static IQueryable<T> CustomInclude<T, TProperty>(this IQueryable<T> query,
Expression<Func<T,TProperty>> include) where T : class
{
TestDB.Includes.Add(include);
return query.Include(include);
}
}
You could also 'override' the normal Include method from System.Data.Entity.
I say 'override', because technically it's not really possible to override an extension method, but you can just create an extension method called Include with the same parameters yourself, and if you don't include System.Data.Entity where you use it, there's no ambiguity between your own method and the one from System.Data.Entity:
public static class EntityExtensions
{
public static IQueryable<T> Include<T, TProperty>(this IQueryable<T> query,
Expression<Func<T,TProperty>> include) where T : class
{
TestDB.Includes.Add(include);
var method = typeof(QueryableExtensions)
.GetMethods(BindingFlags.Public | BindingFlags.Static)
.Where(m => m.Name == "Include")
.First(m => m.GetParameters().All(p => p.ParameterType.IsGenericType));
var generic = method.MakeGenericMethod(typeof(T), typeof(TProperty));
return (IQueryable<T>)generic.Invoke(query, new object[] { query, include });
}
}

I write here another answer to your question (but not the solution you need).
You can retrieve the objects added to entity framework 6 include list from an already retrieved entity in the same way the entity proxy does.
The property to retrieve if a property should be lazy loaded on access (so not already loaded and not in include list) is Relationship.IsLoaded. You can find the list of relationships in YourEntityWithProxy._entityWrapper.Relationships.
_entityWrapper and other properties are private so you need to use reflection to read them.

With the help of Alex, I was able to get what I wanted.
First, to get the name of the includes, I used a small variation of one of the earlier versions of the answer Alex posted:
internal static class IncludeVisitorExtensions
{
public static object GetPrivatePropertyValue( this object obj, string propName )
{
PropertyInfo propertyInfo = obj.GetType().GetProperty( propName, BindingFlags.Public
| BindingFlags.NonPublic | BindingFlags.Instance );
return propertyInfo.GetValue( obj, null );
}
public static object GetPrivateFieldValue( this object obj, string fieldName )
{
FieldInfo fieldInfo = obj.GetType().GetField( fieldName, BindingFlags.Public
| BindingFlags.NonPublic | BindingFlags.Instance );
return fieldInfo?.GetValue( obj );
}
}
internal class IncludeVisitor : ExpressionVisitor
{
private static readonly IncludeVisitor Visitor;
private static List<string> _includes;
private IncludeVisitor() { }
static IncludeVisitor()
{
Visitor = new IncludeVisitor();
}
public static ICollection<string> GetIncludes( Expression expr )
{
_includes = new List<string>();
Visitor.Visit( expr );
return _includes;
}
protected override Expression VisitMethodCall( MethodCallExpression node )
{
if (node.Method.Name != "Include" && node.Method.Name != "IncludeSpan")
return base.VisitMethodCall( node );
//"Include" == .Where() is present in the query
//"IncludeSpan" == no .Where() in the query
try
{
if (node.Method.Name == "Include")
{
string includedObjectName = (string) node.Arguments.First().GetPrivatePropertyValue("Value");
if (includedObjectName != null)
{
_includes.Add(includedObjectName);
}
}
else if (node.Method.Name == "IncludeSpan")
{
var spanList =
node.Arguments.First().GetPrivatePropertyValue("Value").GetPrivatePropertyValue("SpanList");
var navigations = ((IEnumerable<object>) spanList).Select(s => s.GetPrivateFieldValue("Navigations"));
foreach (var nav in navigations)
_includes.Add(string.Join(".", (IEnumerable<string>) nav));
}
}
catch (Exception e) { }
return base.VisitMethodCall( node );
}
}
One little detail I found when testing his code is the difference in how the included tables are found in the expression based on the presence of a .Where() in the IQueryable<>. Thankfully, this can be checked based on the method name, and while the code is a bit ugly, it does dance around the vastly different structures to return the correct name of the table.
Now I have the name of the table, pluralized, as a string. This is because the name is from the DbContext, so I can reflect over the properties and get the Type of the table:
List<PropertyInfo> contextProperties = typeof( TContext ).GetProperties().ToList();
PropertyInfo prop = contextProperties.First( x => x.Name == s );
With the Type, I can accurately find the table I need via Navigation Properties, then I can build a string to send into my new .Include:
ICollection<string> includes = IncludeVisitor.GetIncludes( query.Expression );
foreach (string include in includes)
{
//sometimes the returned include string will be two tables joined with a '.'; these need to be split and each one checked independently
List<string> split = include.Split( '.' ).ToList();
foreach (string s in split)
{
//using .First here because we expect the property to exist
PropertyInfo prop = contextProperties.First( x => x.Name == s );
//the property will be of type DbSet<ObjectType>, so grab the first generic argument (in this case, the object type)
Type dbSetPropertyType = prop.PropertyType.GetGenericArguments().First();
//Get the type we're looking to add in the .Include
var targetTable = GetTargetTableBasedOnTypeViaNavigation(dbSetPropertyType);
//get the name of the property based on the type of the table we just looked up
PropertyInfo contextProperty = contextProperties.SingleOrDefault(x => x.PropertyType.IsGenericType && x.PropertyType.GetGenericArguments().First().Name == targetTable.Name );
string includeString = "";
//build the string and add it to the query
includeString += include + "." + contextPropertyForChangeTracker.Name;
query = query.Include(includeString);
}
}
This worked on my initial test data sets, though I'm not sure how well it will handle more complex graphs.

How to define anonymous method types to build dynamic queries with LINQ?

I'm busy with a LINQ to SQL project that basically creates multiple threads for each entity type in my database, which constantly queries information from the DB in a thread.
Here's a pseudo example:
streamer.DefineDataExpression<Contacts>(x => x.FirstName == "Bob");
while(true)
{
List<Contacts> MyContactsResult = streamer.ResultList;
// do whatever with MyContactsResult
}
The above code doesn't exist, but this is what I have so far for the 'streamer' class (it obviously doesn't work, but you can see what I'm trying to achieve above):
public void DefineExpression(System.Linq.Expressions.Expression<System.Func<T, bool>> expression)
{
using (var db = new LINQDataContext())
{
ResultList = db.GetTable<T>().Where(expression);
}
}
How do I go about creating a method like 'DefineExpression' that will allow me to query a LINQ type dynamically?

Why not use the Dynamic LINQ provider, as mentioned by Scott Guthrie. I think that would give you everything you are looking for, because you can define the query as a string. Therefore, you can more easily build a string representation of your query, and execute on the fly.

How to create a custom property in a Linq-to-SQL entity class?

I have two tables Studies and Series. Series are FK'd back to Studies so one Study contains a variable number of Series.
Each Series item has a Deleted column indicating it has been logically deleted from the database.
I am trying to implement a Deleted property in the Study class that returns true only if all the contained Series are deleted.
I am using O/R Designer generated classes, so I added the following to the user modifiable partial class for the Study type:
public bool Deleted
{
get
{
var nonDeletedSeries = from s in Series
where !s.Deleted
select s;
return nonDeletedSeries.Count() == 0;
}
set
{
foreach (var series in Series)
{
series.Deleted = value;
}
}
}
This gives an exception "The member 'PiccoloDatabase.Study.Deleted' has no supported translation to SQL." when this simple query is executed that invokes get:
IQueryable<Study> dataQuery = dbCtxt.Studies;
dataQuery = dataQuery.Where((s) => !s.Deleted);
foreach (var study in dataQuery)
{
...
}
Based on this http://www.foliotek.com/devblog/using-custom-properties-inside-linq-to-sql-queries/, I tried the following approach:
static Expression<Func<Study, bool>> DeletedExpr = t => false;
public bool Deleted
{
get
{
var nameFunc = DeletedExpr.Compile();
return nameFunc(this);
}
set
{ ... same as before
}
}
I get the same exception when a query is run that there is no supported translation to SQL. (
The logic of the lambda expression is irrelevant yet - just trying to get past the exception.)
Am I missing some fundamental property or something to allow translation to SQL? I've read most of the posts on SO about this exception, but nothing seems to fit my case exactly.

I believe the point of LINQ-to-SQL is that your entities are mapped for you and must have correlations in the database. It appears that you are trying to mix the LINQ-to-Objects and LINQ-to-SQL.
If the Series table has a Deleted field in the database, and the Study table does not but you would like to translate logical Study.Deleted into SQL, then extension would be a way to go.
public static class StudyExtensions
{
public static IQueryable<study> AllDeleted(this IQueryable<study> studies)
{
return studies.Where(study => !study.series.Any(series => !series.deleted));
}
}
class Program
{
public static void Main()
{
DBDataContext db = new DBDataContext();
db.Log = Console.Out;
var deletedStudies =
from study in db.studies.AllDeleted()
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
}
}
This maps your "deleted study" expression into SQL:
SELECT t0.study_id, t0.name
FROM study AS t0
WHERE NOT EXISTS(
SELECT NULL AS EMPTY
FROM series AS t1
WHERE (NOT (t1.deleted = 1)) AND (t1.fk_study_id = t0.study_id)
)
Alternatively you could build actual expressions and inject them into your query, but that is an overkill.
If however, neither Series nor Study has the Deleted field in the database, but only in memory, then you need to first convert your query to IEnumerable and only then access the Deleted property. However doing so would transfer records into memory before applying the predicate and could potentially be expensive. I.e.
var deletedStudies =
from study in db.studies.ToList()
where study.Deleted
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}

When you make your query, you will want to use the statically defined Expression, not the property.
Effectively, instead of:
dataQuery = dataQuery.Where((s) => !s.Deleted);
Whenever you are making a Linq to SQL query, you will instead want to use:
dataQuery = dataQuery.Where(DeletedExpr);
Note that this will require that you can see DeletedExpr from dataQuery, so you will either need to move it out of your class, or expose it (i.e. make it public, in which case you would access it via the class definition: Series.DeletedExpr).
Also, an Expression is limited in that it cannot have a function body. So, DeletedExpr might look something like:
public static Expression<Func<Study, bool>> DeletedExpr = s => s.Series.Any(se => se.Deleted);
The property is added simply for convenience, so that you can also use it as a part of your code objects without needing to duplicate the code, i.e.
var s = new Study();
if (s.Deleted)
...

Predicates and Lists

I have a generic list.
Some elements of this list belong to a parent element. I retrieved all these elements from a database and i want to recursively build a tree with them.
So, here's what i'm thinking:
Here is my predicate:
public static bool FindChildren(Int32 parentId,CategoryMapping catMapping)
{
if (catMapping.parentId == parentId)
{
return true;
}
else
{
return false;
}
}
root = list[0];
root.childrenElements = root.FindAll(FindChildren(root.id,???)
I can't figure out how this would work. How can i do this kind of predicate?
PS: I'm using VS2005 :(

Try
root.childrenElements =
root
.Where( i => i.parentId == yourCatMapping.parentId)
.ToArray();
EDIT
In .net 2.0 I think it is
root.FindAll(
delegate(CategoryMapping mapping)
{
return mapping.parentId == root.Id;
});

You need to specify a delegate to pass to FindAll, rather than a direct function call
(assuming root is List<CategoryMapping>)
root.childrenElements = root.FindAll(c => FindChildren(root.id, c));

You should check out this thread that I started on Forming good predicate delegates to Find() or FindAll() in a List for C# / .NET 2.0
It answers your question pretty clearly.

Gregoire's answer is the best, because it:
Doesn't use LINQ (the asker is using VS 2005)
Doesn't use a lambda expression (again, VS 2005)
That said, why not make things (slightly) easier for yourself by writing a function to generate your Predicate for you:
public static Predicate<CategoryMapping> GetIsChildOf(int parentId) {
return delegate(CategoryMapping cm) {
return cm.parentId == parentId;
};
}
Then if you have a List<CategoryMapping> and you want to find all elements with a certain parentId property, you can just call:
root = list[0];
root.childrenElements = list.FindAll(GetIsChildOf(root.id));