Develop Reference

Entity Framework Core Invoke an Expression on Navigation property

Tried to search a lot for this but couldn't come up with an answer that works. Here's what I am trying to do:
I have Entity ObjectA that has a Navigation property ObjectB (not a Collection type, it's a virtual property that gets loaded via lazy loading). When I do a Where query for A, I want to also expand property B in the expression using another Expression that maps B.
Here's some code to demonstrate, question is in the ToObjectADto() function
public static Expression<Func<ObjectB, ObjectBDto>> ToObjectBDto()
{
return b => new ObjectBDto
{
Prop1 = b.Prop1,
Prop2 = b.Prop2;
};
}
public static Expression<Func<ObjectA, ObjectADto>> ToObjectADto()
{
return a => new ObjectADto
{
Name = a.Name,
SomeProperty = a.SomeProperty,
ObjectB = /* How can I call the ToObjectBDto Expression here without re-writing it? */
};
}
var aDto = _dbContext.ObjectAs.Where(q => q.SomeProperty > 0).Select(ToObjectADto());
I tried to create a compiled Expression:
private static _toBDtoCompiled = ToObjectBDto().Compile();
and then call it in ToObjectADto() like below but I get the API Error There is already an open DataReader associated error because it's doing it client side.
public static Expression<Func<ObjectA, ObjectADto>> ToObjectADto()
{
return a => new ObjectADto
{
Name = a.Name,
SomeProperty = a.SomeProperty,
ObjectB = _toBDto().Invoke(a.ObjectB)
};
}

My recommendation would be to save yourself the work and leverage AutoMapper. The benefit here is that Automapper can feed off EF's IQueryable implementation via ProjectTo to build queries and populate DTO graphs.
var config = new MapperConfiguration(cfg =>
{
cfg.CreateMap<ObjectA, ObjectADto>();
cfg.CreateMap<ObjectB, ObjectBDto>();
});
var aDto = _dbContext.ObjectAs.Where(q => q.SomeProperty > 0).ProjectTo<ObjectADto>(config);
Any particular mapping that cannot be inferred between Object and DTO can be set up in the mapping. The benefit of ProjectTo vs. custom mapping is that it will build a related query without tripping out lazy load hits or risking triggering code that EF cannot translate into SQL. (One query to populate all relevant DTOs)
Automapper can assist with copying values from a DTO back into either a new entity or update an existing entity:
var config = new MapperConfiguration(cfg =>
{
cfg.CreateMap<NewObjectADto, ObjectA>();
cfg.CreateMap<UpdateObjectADto, ObjectA>();
});
var mapper = config.CreateMapper();
New..
var objectA = mapper.Map<ObjectA>(dto);
_dbContext.ObjectAs.Add(objectA);
... or Update existing.
var objectA = _dbContext.ObjectAs.Single(x => x.ObjectAId == dto.ObjectAId);
mapper.Map(objectA, dto);
Where the DTO reflects either the data needed to create a new object, or the data that the client is allowed to update for updating the existing object. The goal being to keep update/add/delete operations as atomic as possible vs. passing large complex objects /w relatives to be updated all at once. I.e. actions like "AddObjectBToA" "RemoveObjectBFromA" etc. rather than resolving all operations via a single "UpdateObjectA".

It's a pity that C# doesn't handle compiling a lambda into an expression, where one expression calls another. Particularly since an expression tree can represent this case. But then EF Core 3 or higher won't look through invoke expressions anyway.
Automapper is probably easier. But if you don't want to use 3rd party code, you'll have to inline the expression yourself. Including replacing any ParameterExpression with the argument to the method.
public static R Invoke<T, R>(this Expression<Func<T, R>> expression, T argument) => throw new NotImplementedException();
// etc for expressions with more parameters
public class InlineVisitor : ExpressionVisitor {
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method.Name == "Invoke"
&& node.Object == null
&& node.Arguments.Count >= 1
&& node.Arguments[0] is LambdaExpression expr)
return Visit(
new ReplacingExpressionVisitor(
expr.Parameters.ToArray(),
node.Arguments.Skip(1).ToArray())
.Visit(expr.Body)
);
return base.VisitMethodCall(node);
}
}
// usage;
public static Expression<Func<ObjectA, ObjectADto>> ToObjectADto()
{
var ToBorNotToB = ToObjectBDto();
Expression<Func<ObjectA, ObjectADto>> expr = a => new ObjectADto
{
Name = a.Name,
SomeProperty = a.SomeProperty,
ObjectB = ToBorNotToB.Invoke(a.ObjectB)
};
return new InlineVisitor().VisitAndConvert(expr), "");
}

Entity Framework generic expression to selectively hide property

I would like to selectively ignore a property from a table.
I have an API which exposes the following methods.
public interface IReadService
{
FullDTO Get();
HeaderDTO[] GetList();
}
My data structure looks like so:
public ServiceDTO : ServiceHeaderDTO
{
public string LargeXMLData { get; set; }
}
public ServiceHeaderDTO
{
public int Id { get; set; }
public string Description { get; set; }
//.... Other properties
}
I have a few services which have similar issues, So I would like to be able to ignore the XML property in some cases, so I'm not using extra time to send a large string property which will be ignored.
Normally you might write something like this to hide a property
var entities = context.Services.Select(x =>
new Service { Id = Id, Description = Description, LargeXMLData = "" }).ToArray();
var dtos = this.AdaptToDTO(entities);
Now this would be fine if I had to do this in a single service, but when you have 20 services duplicating the logic it gets annoying.
I would like the be able to just say:
var entities = context.Services.Excluding(x => x.LargeXMLData).ToArray();
var dtos = this.AdaptToHeaderDTO(entities);
Edit: I'm not using automapper. Alot of our code has mappings which cannot translate to expressions. I do not want to have to specify maps
Is there a simple way I can exclude a property from a query? Without having to manually build maps.
Preferably a way which uses the existing mappings internal to EF which maps the entity to the db object

Normally you might write something like this to hide a property
var entities = context.Services.Select(x =>
new Service { Id = Id, Description = Description, LargeXMLData = "" })
If you can do that manually, it should be doable automatically using the exact same concept, with little reflection and Expression APIs.
But note that this woult work only for EF Core, since EF6 does not support projecting to entity types, like new Service { ... } here, and projecting to dynamic types at runtime is not trivial and also will break the DTO mapping.
With that being said, following is a sample implementation of the aforementioned concept:
public static partial class QueryableExtensions
{
public static IQueryable<T> Excluding<T>(this IQueryable<T> source, params Expression<Func<T, object>>[] excludeProperties)
{
var excludeMembers = excludeProperties
.Select(p => ExtractMember(p.Body).Name)
.ToList();
if (excludeMembers.Count == 0) return source;
// Build selector like (T e) => new T { Prop1 = e.Prop1, Prop2 = e.Prop2, ... }
// for each public property of T not included in the excludeMembers list,
// which then will be used as argument for LINQ Select
var parameter = Expression.Parameter(typeof(T), "e");
var bindings = typeof(T).GetProperties()
.Where(p => p.CanWrite && !excludeMembers.Contains(p.Name))
.Select(p => Expression.Bind(p, Expression.MakeMemberAccess(parameter, p)));
var body = Expression.MemberInit(Expression.New(typeof(T)), bindings);
var selector = Expression.Lambda<Func<T, T>>(body, parameter);
return source.Select(selector);
}
static MemberInfo ExtractMember(Expression source)
{
// Remove Convert if present (for value type properties cast to object)
if (source.NodeType == ExpressionType.Convert)
source = ((UnaryExpression)source).Operand;
return ((MemberExpression)source).Member;
}
}
The usage would be exactly as desired:
var entities = context.Services.Excluding(x => x.LargeXMLData).ToArray();
The problem with this though is that it will automatically "include" navigation properties and/or unmapped properties.
So it would be better to use EF model metadata instead of reflection. The problem is that currently EF Core does not provide a good public way of plugging into their infrastructure, or to get access to DbContext (thus Model) from IQueryble, so it has to be passed as argument to the custom method:
public static IQueryable<T> Excluding<T>(this IQueryable<T> source, DbContext context, params Expression<Func<T, object>>[] excludeProperties)
{
var excludeMembers = excludeProperties
.Select(p => ExtractMember(p.Body).Name)
.ToList();
if (excludeMembers.Count == 0) return source;
// Build selector like (T e) => new T { Prop1 = e.Prop1, Prop2 = e.Prop2, ... }
// for each property of T not included in the excludeMembers list,
// which then will be used as argument for LINQ Select
var parameter = Expression.Parameter(typeof(T), "e");
var bindings = context.Model.FindEntityType(typeof(T)).GetProperties()
.Where(p => p.PropertyInfo != null && !excludeMembers.Contains(p.Name))
.Select(p => Expression.Bind(p.PropertyInfo, Expression.MakeMemberAccess(parameter, p.PropertyInfo)));
var body = Expression.MemberInit(Expression.New(typeof(T)), bindings);
var selector = Expression.Lambda<Func<T, T>>(body, parameter);
return source.Select(selector);
}
which makes the usage not so elegant (but doing the job):
var entities = context.Services.Excluding(context, x => x.LargeXMLData).ToArray();
Now the only remaining potential problem are shadow properties, but they cannot be handled with projection, so this technique simply cannot be used for entities with shadow properties.
Finally, the pure EF Core alternative of the above is to put the LargeXMLData into separate single property "entity" and use table splitting to map it to the same table. Then you can use the regular Include method to include it where needed (by default it would be excluded).

I needed to double-check this before answering, but are you using Automapper or some other mapping provider for the ProjectTo implementation? Automapper's ProjectTo extension method requires a mapper configuration, so it may be that your mapping implementation is materializing the entities prematurely.
With Automapper, your example projecting to a DTO that does not contain the large XML field would result in a query to the database that does not return the large XML without needing any new "Exclude" method.
For instance, if I were to use:
var config = new MappingConfiguration<Service, ServiceHeaderDTO>();
var services = context.Services
.ProjectTo<ServiceHeaderDTO>(config)
.ToList();
The resulting SQL would not return the XMLData because ServiceHeaderDTO does not request it.
It is equivalent to doing:
var services = context.Services
.Select(x => new ServiceHeaderDTO
{
ServiceId = x.ServiceId,
// ... remaining fields, excluding the XML Data
}).ToList();
As long as I don't reference x.LargeXMLData, it will not be returned by my resulting query. Where you can run into big data coming back is if something like the following happens behind the scenes:
var services = context.Services
.ToList()
.Select(x => new ServiceHeaderDTO
{
ServiceId = x.ServiceId,
// ... remaining fields, excluding the XML Data
}).ToList();
That extra .ToList() call will materialize the complete Service entity to memory including the XMLData field. Now Automapper's ProjectTo does not work against IEnumerable, only IQueryable so it is unlikely that any query fed to it was doing this, but if you are using a home-grown mapping implementation where ProjectTo is materializing the entities before mapping, then I would strongly recommend using Automapper as it's IQueryable implementation avoids this problem for you automatically.
Edit: Tested with EF Core and Automapper just in case the behaviour changed, but it also excludes anything not referenced in the mapped DTO.

NHibernate Like restriction on nested string properties

For simplicity, let's guess there are two entities:
public class Entity
{
public string Value { get; set; }
public ChildEntity Child { get; set; }
}
public class ChildEntity
{
public string Value { get; set; }
}
I need to find all entities where either Value or Child.Value are insensitive like specified string query.
That's what I have by now:
Entity entity = null;
ChildEntity child = null;
var nhibernateQuery = session
.QueryOver(() => entity)
.JoinAlias(() => entity.Child, () => child);
if (!string.IsNullOrWhiteSpace(query))
{
nhibernateQuery = nhibernateQuery
.Where(
Restrictions.Or(
Restrictions.On(() => entity).IsInsensitiveLike(query),
Restrictions.On(() => child).IsInsensitiveLike(query)
)
);
}
return nhibernateQuery.List().ToArray();
I get the NullReferenceException - it seems like Restrictions.On does not handle alias correctly.
Another approach that I have tried is .JoinQueryOver() which is suggested by this post:
return session
.QueryOver<Entity>()
.Where(Restrictions.InsensitiveLike("Value", query))
.JoinQueryOver(e => e.Child)
.Where(Restrictions.InsensitiveLike("Value", query));
This thing works except for one thing: it returns all items where both Value and Child.Value are like query. I need the same thing, but with or logic.
What should be done to make it work? I would like to use .QueryOver(), either with or without aliases, but without .CreateCriteria(), but will appreciate if you help me with any working solution.

The problem has been solved by using NHibernate LINQ .Query<>().
It can resolve all joins and relations by itself.
At the same time, .Contains() method is translated into case-insensitive LIKE statement for MS SQL which suits me needs.
var nhibernateQuery = session
.Query<Entity>();
if (!string.IsNullOrWhiteSpace(query))
{
nhibernateQuery = nhibernateQuery
.Where(e => e.Value.Contains(query) || e.Child.Value.Contains(query));
}
return nhibernateQuery.ToArray();

Static expression as instance-property for dynamic linq-queries

I'm using a modified version of LinqKit in order to have my extensions at one place.
Therefore I have a singel part of each of my entity-classes where I define my expression, e.g. for tblMain:
public partial class tblMain
{
public static Expression<Func<tblMain, bool>> IsVisible => (e) => e.MainStatus == "Visible";
}
In a query I am now able to write something like this
var visibleEntries = dbContext
.tblMain
.AsExpandable()
.Where(m => tblMain.IsVisible.Invoke(m))
.ToList();
which will return me all of the visible entries of the table tblMain.
I wondered if there is any way to not have this a static property. This would enable me to use Interfaces like IVisibilityEntity to force a public IsVisible property on specific types.
For now I've ended up with:
public Expression<Func<bool>> IsVisible2 => Expression.Lambda<Func<bool>>(Expression.Invoke(IsVisible, Expression.Variable(typeof(tblMain))));
but this throws me an exception
InvalidOperationException: variable 'm' of type 'tblMain' referenced from scope '', but it is not defined.
Same when using Expression.Constant(this, typeof(tblMain)) as second parameter.
What I would love to have is a query like
var visibleEntries = dbContext
.tblMain
.AsExpandable()
.Where(m => m.IsVisible.Invoke())
.ToList();
This may seem not that much of a change. But I really want to be able to use the interface-feature to describe my underlying database-model.
With interfaces it also allows checks, e.g. if(myEntity is IVisibilityEntity) to do specific visibility-stuff.
Any ideas if this is possible and how this can be achieved?
Edit 1
As of the first comment. I use LinqKit to enable the same logic for sub-queries:
var visibleEntries = dbContext
.tblMain
.AsExpandable()
.Where(m => tblMain.IsVisible.Invoke(m))
.Select(m => new
{
VisibleSubs = m.tblSub.Where(s => tblSub.IsVisible.Invoke(s)).Select(s => new
{
// ...
})
})
.ToList();
The query above would give me all visible main-entries and their related (and also visible) sub-entries. But this is not possible with simply writing m.tblSub.Where(tblSub.IsVisible) as this shows
CS1929 'ICollection<tblSub>' does not contain a definition for 'Where' and the best extension method overload 'Queryable.Where<tblSub>(IQueryable<tblSub>, Expression<Func<tblSub, bool>>)' requires a receiver of type 'IQueryable<tblSub>'

what about defining your properties like this: https://damieng.com/blog/2009/06/24/client-side-properties-and-any-remote-linq-provider
It is essentially property definition in the form of:
partial class Employee {
private static readonly CompiledExpression<Employee,string> fullNameExpression
= DefaultTranslationOf<Employee>.Property(e => e.FullName).Is(e => e.Forename + " " + e.Surname);
private static readonly CompiledExpression<Employee,int> ageExpression
= DefaultTranslationOf<Employee>.Property(e => e.Age).Is(e => DateTime.Now.Year - e.BirthDate.Value.Year - (((DateTime.Now.Month < e.BirthDate.Value.Month) || (DateTime.Now.Month == e.BirthDate.Value.Month && DateTime.Now.Day < e.BirthDate.Value.Day)) ? 1 : 0)));
public string FullName {
get { return fullNameExpression.Evaluate(this); }
}
public int Age {
get { return ageExpression.Evaluate(this); }
}
}
and the use the Microsoft.Linq.Translations package to resolve.

A better way to use AutoMapper to flatten nested objects?

I have been flattening domain objects into DTOs as shown in the example below:
public class Root
{
public string AParentProperty { get; set; }
public Nested TheNestedClass { get; set; }
}
public class Nested
{
public string ANestedProperty { get; set; }
}
public class Flattened
{
public string AParentProperty { get; set; }
public string ANestedProperty { get; set; }
}
// I put the equivalent of the following in a profile, configured at application start
// as suggested by others:
Mapper.CreateMap<Root, Flattened>()
.ForMember
(
dest => dest.ANestedProperty
, opt => opt.MapFrom(src => src.TheNestedClass.ANestedProperty)
);
// This is in my controller:
Flattened myFlattened = Mapper.Map<Root, Flattened>(myRoot);
I have looked at a number of examples, and so far this seems to be the way to flatten a nested hierarchy. If the child object has a number of properties, however, this approach doesn't save much coding.
I found this example:
http://consultingblogs.emc.com/owainwragg/archive/2010/12/22/automapper-mapping-from-multiple-objects.aspx
but it requires instances of the mapped objects, required by the Map() function, which won't work with a profile as I understand it.
I am new to AutoMapper, so I would like to know if there is a better way to do this.

I much prefer avoiding the older Static methods and do it like this.
Place our mapping definitions into a Profile. We map the Root first, and then apply the mappings of the Nested afterwards. Note the use of the Context.
public class MappingProfile : Profile
{
public MappingProfile()
{
CreateMap<Root, Flattened>()
.AfterMap((src, dest, context) => context.Mapper.Map(src.TheNestedClass, dest));
CreateMap<Nested, Flattened>();
}
}
The advantage of defining both the mapping from Root to Flattened and Nested to Flatterned is that you retain full control over the mapping of the properties, such as if the destination property name is different or you want to apply a transformation etc.
An XUnit test:
[Fact]
public void Mapping_root_to_flattened_should_include_nested_properties()
{
// ARRANGE
var myRoot = new Root
{
AParentProperty = "my AParentProperty",
TheNestedClass = new Nested
{
ANestedProperty = "my ANestedProperty"
}
};
// Manually create the mapper using the Profile
var mapper = new MapperConfiguration(cfg => cfg.AddProfile(new MappingProfile())).CreateMapper();
// ACT
var myFlattened = mapper.Map<Root, Flattened>(myRoot);
// ASSERT
Assert.Equal(myRoot.AParentProperty, myFlattened.AParentProperty);
Assert.Equal(myRoot.TheNestedClass.ANestedProperty, myFlattened.ANestedProperty);
}
By adding AutoMapper's serviceCollection.AddAutoMapper() from the AutoMapper.Extensions.Microsoft.DependencyInjection nuget package to your start up, the Profile will be picked up automatically, and you can simply inject IMapper into wherever you are applying the mapping.

In the latest version of AutoMapper, there's a naming convention you can use to avoid multiple .ForMember statements.
In your example, if you update your Flattened class to be:
public class Flattened
{
public string AParentProperty { get; set; }
public string TheNestedClassANestedProperty { get; set; }
}
You can avoid the use of the ForMember statement:
Mapper.CreateMap<Root, Flattened>();
Automapper will (by convention) map Root.TheNestedClass.ANestedProperty to Flattened.TheNestedClassANestedProperty in this case. It looks less ugly when you're using real class names, honest!

2 more possible solutions:
Mapper.CreateMap<Nested, Flattened>()
.ForMember(s=>s.AParentProperty, o=>o.Ignore());
Mapper.CreateMap<Root, Flattened>()
.ForMember(d => d.ANestedProperty, o => o.MapFrom(s => s.TheNestedClass));
An alternative approach would be the below, but it would not pass the Mapper.AssertConfigurationIsValid().
Mapper.CreateMap<Nested, Flattened>()
//.ForMember map your properties here
Mapper.CreateMap<Root, Flattened>()
//.ForMember... map you properties here
.AfterMap((s, d) => Mapper.Map(s.TheNestedClass, d));

Not sure if this adds value to the previous solutions, but you could do it as a two-step mapping. Be careful to map in correct order if there are naming conflicts between the parent and child (last wins).
Mapper.CreateMap<Root, Flattened>();
Mapper.CreateMap<Nested, Flattened>();
var flattened = new Flattened();
Mapper.Map(root, flattened);
Mapper.Map(root.TheNestedClass, flattened);

To improve upon another answer, specify MemberList.Source for both mappings and set the nested property to be ignored. Validation then passes OK.
Mapper.Initialize(cfg =>
{
cfg.CreateMap<SrcNested, DestFlat>(MemberList.Source);
cfg.CreateMap<SrcRoot, DestFlat>(MemberList.Source)
.ForSourceMember(s => s.Nested, x => x.Ignore())
.AfterMap((s, d) => Mapper.Map(s.Nested, d));
});
Mapper.AssertConfigurationIsValid();
var dest = Mapper.Map<SrcRoot, DestFlat>(src);

I wrote extension method to solve similar problem:
public static IMappingExpression<TSource, TDestination> FlattenNested<TSource, TNestedSource, TDestination>(
this IMappingExpression<TSource, TDestination> expression,
Expression<Func<TSource, TNestedSource>> nestedSelector,
IMappingExpression<TNestedSource, TDestination> nestedMappingExpression)
{
var dstProperties = typeof(TDestination).GetProperties().Select(p => p.Name);
var flattenedMappings = nestedMappingExpression.TypeMap.GetPropertyMaps()
.Where(pm => pm.IsMapped() && !pm.IsIgnored())
.ToDictionary(pm => pm.DestinationProperty.Name,
pm => Expression.Lambda(
Expression.MakeMemberAccess(nestedSelector.Body, pm.SourceMember),
nestedSelector.Parameters[0]));
foreach (var property in dstProperties)
{
if (!flattenedMappings.ContainsKey(property))
continue;
expression.ForMember(property, opt => opt.MapFrom((dynamic)flattenedMappings[property]));
}
return expression;
}
So in your case it can be used like this:
var nestedMap = Mapper.CreateMap<Nested, Flattened>()
.IgnoreAllNonExisting();
Mapper.CreateMap<Root, Flattened>()
.FlattenNested(s => s.TheNestedClass, nestedMap);
IgnoreAllNonExisting() is from here.
Though it's not universal solution it should be enough for simple cases.
So,
You don't need to follow flattening convention in destination's properties
Mapper.AssertConfigurationIsValid() will pass
You can use this method in non-static API (a Profile) as well

I created a simple example using AutoMappers new naming convention rules to map from flattened to nested objects, hope this helps
https://dotnetfiddle.net/i55UFK