Develop Reference

Filter DbSet by list of objects

I am using Entity Framework Core with SQL Server to filter a list of division entities by a list of passed in search objects. I want to return division that match the criteria in any one of the search objects. The search object class looks like:
public class SearchObject
{
public int DivisionId { get; set; }
public int Status { get; set; }
}
Here is the query I tried:
var searchObjects = new List<SearchObject> { ... };
IQueryable<Product> query = myContext.Divisions.Where(div =>
searchObjects.Any(searchObj =>
searchObj.Status == div.Status &&
searchObj.DivisionId == div.DivisionId))
.Select(...);
When the IQueryable enumerates, I get an error stating: "The Linq Expresion DbSet ... Could not be translated ..." What I need is something like .Contains(), but that works with a list of SearchObj.

Well you have the right idea by
What I need is something like .Contains(), but that works with a list of SearchObj.
and this is how you'd do it
var searchObjects = new List<SearchObject> { ... };
var searchIds = searchObjects.Select(x => x.Divisiond) //.ToList() perhaps?
var searchStatus = searchObjects.Select(x => x.Status) //.ToList() perhaps?
//not you can simply use .Contains and it should generate a WHERE EXISTS query
IQueryable<Product> query = myContext.Divisions
.Where(div =>
searchIds.Contains(div.DivisionId) &&
searchStatus.Contains(div.Status))
.Select(...);

EF code-first - IQueryable having multiplication on a specific decimal property

I have been struggling with the following problem on Entity Framework code-first.
I have an entity class with a decimal, and I have a multiplier decimal parameter.
I want to build a query (but not to call it), which returns the entities, but the Bar property need to be multiplied with my parameter.
From coding side:
public class Foo
{
public Guid Id { get; set; }
public Decimal Bar { get; set; }
}
// this simple stuff returns the entities after some filterings.
Context.Set<Foo>().Where(x => querying on many props).ToList();
This method is similiar what I want to achieve:
public IQueryable<Foo> GetFooQuery( .. Many properties used to the query .. , Decimal Multiplier)
{
var IQueryablePart = Context.Set<Foo>().Where(querying with the parameters);
/* ... and what to do here? ... */
/* IQueryablePart = IQueryablePart.Select(x => new {
Bar = Bar * Multiplier <-- this is okay
}); */
// but how to retrieve the other columns without listing them one by one, and how to return the data as IQueryable<Foo> ?
return IQueryablePart;
}
I would like to use this method in the following way:
IQueryable<Foo> FullQuery = null;
for(some loop, may be 10 or 1000 iterations, it depends) {
var Part = GetFooQuery(/* .. query params ..*/, 2);
if(MyFullQuery == null)
FullQuery = Part;
else
FullQuery.Union(Part);
}
// and in the end, do the db call once:
var Result = FullQuery.ToList();
In SQL, I would handle it like this:
SELECT
Id,
Bar * #MyValue as Bar,
# and all other columns
FROM
Foo
WHERE
(param queries 1) OR
(param queries 2) OR
---
(param queries N)
My question is: what is the way to do this via IQueryable and EF? The most important, I need to call the db only one time.
I reckon it may be some query building stuff, but I'm not familiar with it yet, any help will be very appreciated.

EF6 does not support projection (select) to a class mapped as entity. Hence the only option you have is to project to some anonymous or special class. For your scenario, the easiest I see is a class like this:
public class FooBar
{
public Foo Foo { get; set; }
public decimal Bar { get; set; }
}
Then the single query method could be like this:
public IQueryable<FooBar> GetFooQuery( .. Many properties used to the query .. , decimal multiplier)
{
return Context.Set<Foo>()
.Where(querying with the parameters)
.Select(foo => new FooBar
{
Foo = foo,
Bar = foo.Bar * multiplier
});
}
Now you can build your full query:
IQueryable<FooBar> fullQuery = null;
for (some loop, may be 10 or 1000 iterations, it depends)
{
var subQuery = GetFooQuery(/* .. query params ..*/, 2);
fullQuery = fullQuery == null ? subquery : fullQuery.Union(subQuery);
}
Note that if you use different multiplier (otherwise the whole procedure does not make sense), you'd better use LINQ Concat method (which translates to SQL UNION ALL) rather then Union (which translates to SQL UNION).
Finally, you can materialize the result as Foo sequennce by executing the single final SQL query, switching to LINQ to Objects and converting the FooBar to Foo like this:
var result = fullQuery.
.AsEnumerable() // db query ends here
.Select(fooBar =>
{
fooBar.Foo.Bar = fooBar.Bar;
return fooBar.Foo;
})
.ToList();

How to create a reusable where clause for EF6

I have recently moved from coding in Java to c# and I am still learning the various elements of c#.
To access an existing database, which I cannot redesign, I am using Entity Frameworks 6 and 'Code First from database' to generate contexts and types representing the database tables. I am using Ling-To-SQL to retrieve the data from the database which is heavily denormalized.
My current task is create a report where each section is read from various tables, which all have a relationship to one base table.
This is my working example:
using(var db = new PaymentContext())
{
var out = from pay in db.Payment
join typ in db.Type on new { pay.ID, pay.TypeID } equals
new { typ.ID, typ.TypeID }
join base in db.BaseTable on
new { pay.Key1, pay.Key2, pay.Key3, pay.Key4, pay.Key5 } equals
new { base.Key1, base.Key2, base.Key3, base.Key4, base.Key5 }
where
base.Cancelled.Equals("0") &&
base.TimeStamp.CompareTo(startTime) > 0 &&
base.TimeStamp.CompareTo(endTime) < 1 &&
.
(other conditions)
.
group new { pay, typ } by new { typ.PaymentType } into grp
select new
{
name = grp.Key,
count = grp.Count(),
total = grp.Sum(x => x.pay.Amount)
};
}
There will be a large number of sections in the report and each section will generate a where clause which will contain the conditions shown. In some sections, the required data will be extracted from tables up to five levels below the BaseTable.
What I want to do is create a resuable where clause for each report section, to avoid a lot of duplicated code.
After a lot of searching, I tried to use the solution suggested here , but this has been superseded in Entity Framework 6.
How do I avoid duplicating code unnecessarily?

I did try to use the extension clauses you suggested, but my generated classes do not extend the BaseTable, so I had to explicitly define the link through the navigation property. As only a small number of tables will be common in the queries, I decided to apply the filters directly to each table as required. I will define these as required.
krillgar suggested moving to straight LINQ syntax, which seems like good advice. We intend to redesign our database in the near future and this will remove some of the SQL dependency. I merged the suggested filters and full LINQ syntax to access my data.
// A class to hold all the possible conditions applied for the report
// Can be applied at various levels within the select
public class WhereConditions
{
public string CancelledFlag { get; set; } = "0"; // <= default setting
public DateTime StartTime { get; set; }
public DateTime EndTime { get; set; }
}
// Class to define all the filters to be applied to any level of table
public static class QueryExtensions
{
public static IQueryable<BaseTable> ApplyCancellationFilter(this IQueryable<BaseTable> base, WhereConditions clause)
{
return base.Where(bse => bse.CancelFlag.Equals(clause.CancelledFlag));
}
public static IQueryable<BaseTable> ApplyTimeFilter(this IQueryable<BaseTable> base, WhereConditions clause)
{
return base.Where(bse => bse.TimeStamp.CompareTo(clause.StartTime) > 0 &&
bse.TimeStamp.CompareTo(clause.EndTime) < 1);
}
}
And the query is composed as follows:
using (var db = new PaymentContext())
{
IEnumerable<BaseTable> filter = db.BaseTable.ApplyCancellationFilter(clause).ApplyTimeFilter(clause);
var result = db.Payment.
Join(
filter,
pay => new { pay.Key1, pay.Key2, pay.Key3, pay.Key4, pay.Key5 },
bse => new { bse.Key1, bse.Key2, bse.Key3, bse.Key4, bse.Key5 },
(pay, bse) => new { Payment = pay, BaseTable = bse }).
Join(
db.Type,
pay => new { pay.Payment.TypeKey1, pay.Payment.TypeKey2 },
typ => new { typ.TypeKey1, typ.TypeKey2 },
(pay, typ) => new { name = typ.Description, amount = pay.Amount }).
GroupBy(x => x.name).
Select(y => new { name = y.Key,
count = y.Count(),
amount = y.Sum(z => z.amount)});
}
And then to finally execute composed query.
var reportDetail = result.ToArray(); // <= Access database here
As this query is the simplest I will have to apply, future queries will become much more complicated.

The nice thing about LINQ is that methods like Where() return an IEnumerable<T> that you can feed into the next method.
You could refactor the where clauses into extension methods akin to:
public static class PaymentQueryExtensions {
public static IQueryable<T> ApplyNotCancelledFilter(
this IQueryable<T> payments)
where T : BaseTable {
// no explicit 'join' needed to access properties of base class in EF Model
return payments.Where(p => p.Cancelled.Equals("0"));
}
public static IQueryable<T> ApplyTimeFilter(
this IQueryable<T> payments, DateTime startTime, DateTime endTime)
where T: BaseTable {
return payments.Where(p => p.TimeStamp.CompareTo(startTime) > 0
&& p.TimeStamp.CompareTo(endTime) < 1);
}
public static IGrouping<Typ, T> GroupByType(
this IQueryable<T> payments)
where T: BaseTable {
// assuming the relationship Payment -> Typ has been set up with a backlink property Payment.Typ
// e.g. for EF fluent API:
// ModelBuilder.Entity<Typ>().HasMany(t => t.Payment).WithRequired(p => p.Typ);
return payments.GroupBy(p => p.Typ);
}
}
And then compose your queries using these building blocks:
IEnumerable<Payment> payments = db.Payment
.ApplyNotCancelledFilter()
.ApplyTimeFilter(startTime, endTime);
if (renderSectionOne) {
payments = payments.ApplySectionOneFilter();
}
var paymentsByType = payments.GroupByType();
var result = paymentsByType.Select(new
{
name = grp.Key,
count = grp.Count(),
total = grp.Sum(x => x.pay.Amount)
}
);
Now that you have composed the query, execute it by enumerating. No DB access has happened until now.
var output = result.ToArray(); // <- DB access happens here
Edit After the suggestion of Ivan, I looked at our codebase. As he mentioned, the Extension methods should work on IQueryable instead of IEnumerable. Just take care that you only use expressions that can be translated to SQL, i.e. do not call any custom code like an overriden ToString() method.
Edit 2 If Payment and other model classes inherit BaseTable, the filter methods can be written as generic methods that accept any child type of BaseTable. Also added example for grouping method.

How to return DbGeography.Distance calculated value In Code First Entity Framework without losing strong typing?

Currently I have an entity that is "geolocatable" via a SqlGeography column that I can use via expressions for filtering and sorting. I am already able to get all entities within distance x of point y and sort by entities closest to (or furthest from) point y. However, in order to return the distance from the entity to y I have to recalculate the distance in the application because I have not yet determined how to materialize the result of the distance calculation from the database to the entities in the IQueryable. This is a mapped entity and a great deal of application logic surrounds the type of entity returned so projecting it into a dynamic object is not a viable option for this implementation (though I understand how that would work). I have also tried using an unmapped object that inherits from the mapped entity but that suffers the same problems. Essentially, as I understand it, I should be able to define the getter of an unmapped property to assign a computed value in a queryable extension IF I modify the expression tree that represents the IQueryable but the how escapes me. I've written expressions in this manner before but I think I need to be able to modify the existing select rather than just chaining on a new Expression.Call which is unexplored territory for me.
The following should code should properly illustrate the problem:
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
using System.Data.Entity.ModelConfiguration;
using System.Data.Entity.Spatial; // from Microsoft.SqlServer.Types (Spatial) NuGet package
using System.Linq;
public class LocatableFoo
{
[Key]
public int Id { get; set; }
public DbGeography Geolocation { get; set; }
[NotMapped]
public double? Distance { get; set; }
}
public class PseudoLocatableFoo : LocatableFoo
{
}
public class LocatableFooConfiguration : EntityTypeConfiguration<LocatableFoo>
{
public LocatableFooConfiguration()
{
this.Property(foo => foo.Id).HasColumnName("id");
this.Property(foo => foo.Geolocation).HasColumnName("geolocation");
}
}
public class ProblemContext : DbContext
{
public DbSet<LocatableFoo> LocatableFoos { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new LocatableFooConfiguration());
base.OnModelCreating(modelBuilder);
}
}
public class Controller
{
public Controller(ProblemContext context) // dependency injection
{
this.Context = context;
}
private ProblemContext Context { get; set; }
/* PROBLEM IN THIS METHOD:
* Do not materialize results (ie ToList) and then calculate distance as is done currently <- double calculation of distance in DB and App I am trying to solve
* Must occur prior to materialization
* Must be assignable to "query" that is to type IQueryable<LocatableFoo>
*/
public IEnumerable<LocatableFoo> GetFoos(decimal latitude, decimal longitude, double distanceLimit)
{
var point = DbGeography.FromText(string.Format("Point({0} {1})", longitude, latitude), 4326); // NOTE! This expects long, lat rather than lat, long.
var query = this.Context.LocatableFoos.AsQueryable();
// apply filtering and sorting as proof that EF can turn this into SQL
query = query.Where(foo => foo.Geolocation.Distance(point) < distanceLimit);
query = query.OrderBy(foo => foo.Geolocation.Distance(point));
//// this isn't allowed because EF doesn't allow projecting to mapped entity
//query = query.Select( foo => new LocatableFoo { Id = foo.Id, Geolocation = foo.Geolocation, Distance = foo.Geolocation.Distance(point) });
//// this isn't allowed because EF doesn't allow projecting to mapped entity and PseudoLocatableFoo is considered mapped since it inherits from LocatableFoo
//query = query.Select( foo => new PseudoLocatableFoo { Id = foo.Id, Geolocation = foo.Geolocation, Distance = foo.Geolocation.Distance(point) });
//// this isn't allowed because we must be able to continue to assign to query, type must remain IQueryable<LocatableFoo>
//query = query.Select( foo => new { Id = foo.Id, Geolocation = foo.Geolocation, Distance = foo.Geolocation.Distance(point) });
// this is what I though might work
query = query.SelectWithDistance(point);
this.Bar(query);
var results = query.ToList(); // run generated SQL
foreach (var result in results) //problematic duplicated calculation
{
result.Distance = result.Geolocation.Distance(point);
}
return results;
}
// fake method representing lots of app logic that relies on knowing the type of IQueryable<T>
private IQueryable<T> Bar<T>(IQueryable<T> foos)
{
if (typeof(T) == typeof(LocatableFoo))
{
return foos;
}
throw new ArgumentOutOfRangeException("foos");
}
}
public static class QueryableExtensions
{
public static IQueryable<T> SelectWithDistance<T>(this IQueryable<T> queryable, DbGeography pointToCalculateDistanceFrom)
{
/* WHAT DO?
* I'm pretty sure I could do some fanciness with Expression.Assign but I'm not sure
* What to get the entity with "distance" set
*/
return queryable;
}
}

What about replacing the line
var results = query.ToList();
with
var results = query
.Select(x => new {Item = x, Distance = x.Geolocation.Distance(point)}
.AsEnumerable() // now you just switch to app execution
.Select(x =>
{
x.Item.Distance = x.Distance; // you don't need to calculate, this should be cheap
return x.Item;
})
.ToList();

The Distance field is logically not part of your table, since it represents a distance to a dynamically specified point. As such it should not be part of your entity.
At this point if you want it being calculated on the db, you should create a Stored procedure, or a TVF (or sg else) that returns your entity extended with the distance. This way you can map the return type to an Entity. It is a clearer design to me btw.

Use Linq Expression object inside and outside query

I'm trying to accept a Func as a parameter to a function, and then use it both inside and outside a Linq query.
Here, idSelector is a Func of some kind which will return a particular SubLedger id in the Transaction object (e.g. t => t.SubLedger1).
public class Transaction {
public int SubLedger1 { get; set; }
public int SubLedger2 { get; set; }
public int SubLedger3 { get; set; }
public decimal Balance { get; set; }
}
public IEnumerable<Transaction> GetSubLedger(DateTime StartDate, Func<Transaction, int> idSelector) {
// simply returns IQueryable of all
DbSet<Transaction> txns = txnRepo.GetAll();
// get opening balance for each sub ledger
var subLedgers = txns.Where(t => t.Date < StartDate)
.GroupBy(idSelector, t => t, (id, txn) => new { ID = id, Balance = txn.Sum(t => t.Amount) })
.ToDictionary(t => t.ID, t => t.Balance);
// fill running balance
var filtered = txns.Where(t => t.Date >= StartDate).ToList();
foreach (var t in filtered)
{
t.Balance = subLedgers[idSelector.Invoke(t)].Balance += t.Amount;
}
return filtered;
}
I need to use idSelector in two places: first in the Linq query to group all transactions into subLedgers, and second to get the running balance for the particular subledger in the filtered results. I realize that Linq requires an Expression<...> instead, but I can't quite figure out how to Invoke that in the second context.
It's possible I'm going about this the wrong way, is there some other way I should try? This question may also be a little muddled, I did try to pare down the code sample as much as possible, so please ask me if anything is unclear.

Use Compile to get the invokeable method from the expression:
t.Balance = subLedgers[idSelector.Compile()(t)].Balance += t.Amount;
(Assuming idSelector is an Expression<Func<Transaction, int>>.)