I have been struggling with the following problem on Entity Framework code-first.
I have an entity class with a decimal, and I have a multiplier decimal parameter.
I want to build a query (but not to call it), which returns the entities, but the Bar property need to be multiplied with my parameter.
From coding side:
public class Foo
{
public Guid Id { get; set; }
public Decimal Bar { get; set; }
}
// this simple stuff returns the entities after some filterings.
Context.Set<Foo>().Where(x => querying on many props).ToList();
This method is similiar what I want to achieve:
public IQueryable<Foo> GetFooQuery( .. Many properties used to the query .. , Decimal Multiplier)
{
var IQueryablePart = Context.Set<Foo>().Where(querying with the parameters);
/* ... and what to do here? ... */
/* IQueryablePart = IQueryablePart.Select(x => new {
Bar = Bar * Multiplier <-- this is okay
}); */
// but how to retrieve the other columns without listing them one by one, and how to return the data as IQueryable<Foo> ?
return IQueryablePart;
}
I would like to use this method in the following way:
IQueryable<Foo> FullQuery = null;
for(some loop, may be 10 or 1000 iterations, it depends) {
var Part = GetFooQuery(/* .. query params ..*/, 2);
if(MyFullQuery == null)
FullQuery = Part;
else
FullQuery.Union(Part);
}
// and in the end, do the db call once:
var Result = FullQuery.ToList();
In SQL, I would handle it like this:
SELECT
Id,
Bar * #MyValue as Bar,
# and all other columns
FROM
Foo
WHERE
(param queries 1) OR
(param queries 2) OR
---
(param queries N)
My question is: what is the way to do this via IQueryable and EF? The most important, I need to call the db only one time.
I reckon it may be some query building stuff, but I'm not familiar with it yet, any help will be very appreciated.
EF6 does not support projection (select) to a class mapped as entity. Hence the only option you have is to project to some anonymous or special class. For your scenario, the easiest I see is a class like this:
public class FooBar
{
public Foo Foo { get; set; }
public decimal Bar { get; set; }
}
Then the single query method could be like this:
public IQueryable<FooBar> GetFooQuery( .. Many properties used to the query .. , decimal multiplier)
{
return Context.Set<Foo>()
.Where(querying with the parameters)
.Select(foo => new FooBar
{
Foo = foo,
Bar = foo.Bar * multiplier
});
}
Now you can build your full query:
IQueryable<FooBar> fullQuery = null;
for (some loop, may be 10 or 1000 iterations, it depends)
{
var subQuery = GetFooQuery(/* .. query params ..*/, 2);
fullQuery = fullQuery == null ? subquery : fullQuery.Union(subQuery);
}
Note that if you use different multiplier (otherwise the whole procedure does not make sense), you'd better use LINQ Concat method (which translates to SQL UNION ALL) rather then Union (which translates to SQL UNION).
Finally, you can materialize the result as Foo sequennce by executing the single final SQL query, switching to LINQ to Objects and converting the FooBar to Foo like this:
var result = fullQuery.
.AsEnumerable() // db query ends here
.Select(fooBar =>
{
fooBar.Foo.Bar = fooBar.Bar;
return fooBar.Foo;
})
.ToList();
Related
Below is my method, which calls a stored procedure.
public List<int> GetActivityListforUser(string userId)
{
IList<int> results = new List<int>();
context.LoadStoredProc("dbo.GetRegionOrganizationActivities")
.WithSqlParam("userId", userId)
.ExecuteStoredProc((handler) =>
{
results = handler.ReadToList<int>().ToList();
});
return results.ToList();
}
My stored procedure dbo.GetRegionOrganizationActivities, returns only one column Id, which is the required result up on passing the parameter userId.
My issue is in the following line:
return results.ToList();
I can see all the list that comes from the stored procedure, but all int values are 0.
The list count matches with stored proc result count, but values should be example: 1,2,3, etc. it shows 0,0,0, etc.
Can any one provide some insight on what I am doing wrong?
It seems you are using the Snickler.EFCore nuget package with EF Core.
Short answer:
You'll have to create a complex object to allow this library to map your result set, something like:
public class RegionOrganizationActivity
{
public int Id { get; set; }
}
then you can use this to extract your list of integers:
public List<int> GetActivityListforUser(string userId)
{
IList<RegionOrganizationActivity> results = new List<RegionOrganizationActivity>();
context.LoadStoredProc("dbo.GetRegionOrganizationActivities")
.WithSqlParam("userId", userId)
.ExecuteStoredProc((handler) =>
{
results = handler.ReadToList<RegionOrganizationActivity>();
});
return results.Select(activity => activity.Id).ToList();
}
Explanation:
handler.ReadToList<int>() won't work here because the ReadToList<T> implementation only supports complex types. It is assuming it needs to create an instance of T and try to match properties to the columns.
Because there are no properties to match when T is int, the latter part fails. This explains why you are getting all values of 0: the only thing the method manages to do is an Activator.CreateInstance<int>() which will return 0.
Their other helper method ReadToValue only supports a single result value, which won't work either.
This means you can't use this library to map your result straight to a List<int>. So you'll need to use complex object to match the result set.
This is my final method.
public List<int> GetActivityListforUser(string userId)
{
List<ActivityId> results = new List<ActivityId>();
context.LoadStoredProc("dbo.GetRegionOrganizationActivities")
.WithSqlParam("userId", userId)
.ExecuteStoredProc((handler) =>
{
results = handler.ReadToList<ActivityId>().ToList();
});
List<int> finalresult = new List<int>();
finalresult = results.Select(a=>a.Id).ToList();
return finalresult.ToList();
}
public class ActivityId
{
public int Id { get; set; }
}
I am in big need of help, i have been trying to do this for some time now.
So I have this Query:
Select name from BlaBlaBla
order by
case when name like '9%' then 1 end,
case when name like '8%' then 1 end,
case when name like '7%' then 1 end,
case when name like '6%' then 1 end,
case when name like '5%' then 1 end,
case when name like '4%' then 1 end,
case when name like '3%' then 1 end,
case when name like '2%' then 1 end,
case when name like '1%' then 1 end,
case when name like '0%' then 1 end,
name
And I want to implement it in a new C#, Asp.Net, class, in my Solution, to the Domain Project, so it will be an OrderType Filter, for some function...
for now I have this:
var param = Expression.Parameter(typeof(T), "item");
var paramName = Expression.Property(param, "Name");
var regexMatch = Expression.Constant("^[0-9]");
var startsWithDigit = Expression.Call(typeof(Regex), "IsMatch",
null, paramName);
var lambda = Expression.Lambda<Func<T, bool>>(startsWithDigit,
param);
return namesList.OrderBy(lambda)
.ThenBy(BlaBla1())
.ThenByDescending(BlaBla2())
.ThenByDescending(BlaBla3())
.ThenBy(BlaBla4());
But it tells me, that Expression does not contain "IsMatch" method.
Can you please help me?
Thank you!!!
The problem here is that expressions containing Regex can't be translated to SQL, so even when you'd succeed in building a correct expression, you can't use it in LINQ to a SQL backend. However, SQL's LIKE method also supports range wildcards like [0-9], so the trick is to make your LINQ translate to SQL containing a LIKE statement.
LINQ-to-SQL offers the possibility to use the SQL LIKE statement explicitly:
return namesList.OrderBy(r => SqlMethods.Like(r.Name, "[0-9]%")) ...
This SqlMethods class can only be used in LINQ-to-SQL though. In Entity Framework there are string functions that translate to LIKE implicitly, but none of them enable the range wildcard ([x-y]). In EF a statement like ...
return namesList.OrderBy(r => r.Name.StartsWith("[0-9]")) ...
... would translate to nonsense:
[Name] LIKE '~[0-9]%' ESCAPE '~'
I.e. it vainly looks for names starting with the literal string "[0-9]". So as long as you keep using LINQ-to-SQL SqlMethods.Like is the way to go.
In Entity Framework 6.1.3 (and lower) we have to use a slightly different way to obtain the same result ...
return namesList.OrderBy(r => SqlFunctions.PatIndex("[0-9]%", c.Name) == 1) ...
... because PatIndex in SqlFunctions also supports range pattern matching.
But in Entity Framwork 6.2 we're back on track with LINQ-to-SQL because of the new DbFunctions.Like function:
return namesList.OrderBy(r => DbFunctions.Like(r.Name, "[0-9]%")) ...
Finally, also Entity Framework core has a Like function:
return namesList.OrderBy(r => EF.Functions.Like(r.Name, "[0-9]%")) ...
Below you see a sample for this kind of way to handle cases for your orderings.
static void Main(string[] args)
{
List<Obvious> list = new List<Obvious>();
for (int i = 0; i < 100; i++)
{
list.Add(new Obvious(i.ToString(), i));
}
string name = list[30].name;
switch (name)
{
case "9":
list.OrderBy(o => o.perc)
.ThenByDescending(o => o.name);
break;
default:
list.OrderByDescending(o => o.name)
.ThenBy(o => o.perc);
break;
}
}
public class Obvious
{
public string name { get; set; }
public int perc { get; set; }
public Obvious(string _name, int _perc)
{
this.name = _name;
this.perc = _perc;
}
}
If I was you I wouldn't try using Expressions to solve this issue since it brings in a lot of complexity.
I see that you would like to have a generic method, so it can work with different domain entities, yet you are expecting that each entity has a Name property.
You can solve this in a more simple way by defining interface that contains Name property. Like this:
public static void Main()
{
var test = new List<YourDomainEntity>()
{
new YourDomainEntity() { Name = "1test", OtherProperty = "1"},
new YourDomainEntity() { Name = "2test", OtherProperty = "2" },
new YourDomainEntity() { Name = "2test", OtherProperty = "1" }
};
var k = Foo(test).ToList();
}
public interface INameOrderable
{
string Name { get; set; }
}
public interface IOtherPropertyOrderable
{
string OtherProperty { get; set; }
}
public static IEnumerable<T> Foo<T>(IEnumerable<T> list) where T : INameOrderable, IOtherPropertyOrderable
{
return list.OrderBy(a => a.Name, new NamesDescComparer()).ThenBy(b => b.OtherProperty);
}
public class NamesDescComparer : IComparer<string>
{
public int Compare(string x, string y) => -String.CompareOrdinal(x, y);
}
class YourDomainEntity : INameOrderable, IOtherPropertyOrderable
{
public string OtherProperty { get; set; }
public string Name { get; set; }
}
I believe the method Foo is what you are looking for.
Note the where T : INameOrderable part. It restricts usage of this method to entities that implement INameOrderable interface
I have recently moved from coding in Java to c# and I am still learning the various elements of c#.
To access an existing database, which I cannot redesign, I am using Entity Frameworks 6 and 'Code First from database' to generate contexts and types representing the database tables. I am using Ling-To-SQL to retrieve the data from the database which is heavily denormalized.
My current task is create a report where each section is read from various tables, which all have a relationship to one base table.
This is my working example:
using(var db = new PaymentContext())
{
var out = from pay in db.Payment
join typ in db.Type on new { pay.ID, pay.TypeID } equals
new { typ.ID, typ.TypeID }
join base in db.BaseTable on
new { pay.Key1, pay.Key2, pay.Key3, pay.Key4, pay.Key5 } equals
new { base.Key1, base.Key2, base.Key3, base.Key4, base.Key5 }
where
base.Cancelled.Equals("0") &&
base.TimeStamp.CompareTo(startTime) > 0 &&
base.TimeStamp.CompareTo(endTime) < 1 &&
.
(other conditions)
.
group new { pay, typ } by new { typ.PaymentType } into grp
select new
{
name = grp.Key,
count = grp.Count(),
total = grp.Sum(x => x.pay.Amount)
};
}
There will be a large number of sections in the report and each section will generate a where clause which will contain the conditions shown. In some sections, the required data will be extracted from tables up to five levels below the BaseTable.
What I want to do is create a resuable where clause for each report section, to avoid a lot of duplicated code.
After a lot of searching, I tried to use the solution suggested here , but this has been superseded in Entity Framework 6.
How do I avoid duplicating code unnecessarily?
I did try to use the extension clauses you suggested, but my generated classes do not extend the BaseTable, so I had to explicitly define the link through the navigation property. As only a small number of tables will be common in the queries, I decided to apply the filters directly to each table as required. I will define these as required.
krillgar suggested moving to straight LINQ syntax, which seems like good advice. We intend to redesign our database in the near future and this will remove some of the SQL dependency. I merged the suggested filters and full LINQ syntax to access my data.
// A class to hold all the possible conditions applied for the report
// Can be applied at various levels within the select
public class WhereConditions
{
public string CancelledFlag { get; set; } = "0"; // <= default setting
public DateTime StartTime { get; set; }
public DateTime EndTime { get; set; }
}
// Class to define all the filters to be applied to any level of table
public static class QueryExtensions
{
public static IQueryable<BaseTable> ApplyCancellationFilter(this IQueryable<BaseTable> base, WhereConditions clause)
{
return base.Where(bse => bse.CancelFlag.Equals(clause.CancelledFlag));
}
public static IQueryable<BaseTable> ApplyTimeFilter(this IQueryable<BaseTable> base, WhereConditions clause)
{
return base.Where(bse => bse.TimeStamp.CompareTo(clause.StartTime) > 0 &&
bse.TimeStamp.CompareTo(clause.EndTime) < 1);
}
}
And the query is composed as follows:
using (var db = new PaymentContext())
{
IEnumerable<BaseTable> filter = db.BaseTable.ApplyCancellationFilter(clause).ApplyTimeFilter(clause);
var result = db.Payment.
Join(
filter,
pay => new { pay.Key1, pay.Key2, pay.Key3, pay.Key4, pay.Key5 },
bse => new { bse.Key1, bse.Key2, bse.Key3, bse.Key4, bse.Key5 },
(pay, bse) => new { Payment = pay, BaseTable = bse }).
Join(
db.Type,
pay => new { pay.Payment.TypeKey1, pay.Payment.TypeKey2 },
typ => new { typ.TypeKey1, typ.TypeKey2 },
(pay, typ) => new { name = typ.Description, amount = pay.Amount }).
GroupBy(x => x.name).
Select(y => new { name = y.Key,
count = y.Count(),
amount = y.Sum(z => z.amount)});
}
And then to finally execute composed query.
var reportDetail = result.ToArray(); // <= Access database here
As this query is the simplest I will have to apply, future queries will become much more complicated.
The nice thing about LINQ is that methods like Where() return an IEnumerable<T> that you can feed into the next method.
You could refactor the where clauses into extension methods akin to:
public static class PaymentQueryExtensions {
public static IQueryable<T> ApplyNotCancelledFilter(
this IQueryable<T> payments)
where T : BaseTable {
// no explicit 'join' needed to access properties of base class in EF Model
return payments.Where(p => p.Cancelled.Equals("0"));
}
public static IQueryable<T> ApplyTimeFilter(
this IQueryable<T> payments, DateTime startTime, DateTime endTime)
where T: BaseTable {
return payments.Where(p => p.TimeStamp.CompareTo(startTime) > 0
&& p.TimeStamp.CompareTo(endTime) < 1);
}
public static IGrouping<Typ, T> GroupByType(
this IQueryable<T> payments)
where T: BaseTable {
// assuming the relationship Payment -> Typ has been set up with a backlink property Payment.Typ
// e.g. for EF fluent API:
// ModelBuilder.Entity<Typ>().HasMany(t => t.Payment).WithRequired(p => p.Typ);
return payments.GroupBy(p => p.Typ);
}
}
And then compose your queries using these building blocks:
IEnumerable<Payment> payments = db.Payment
.ApplyNotCancelledFilter()
.ApplyTimeFilter(startTime, endTime);
if (renderSectionOne) {
payments = payments.ApplySectionOneFilter();
}
var paymentsByType = payments.GroupByType();
var result = paymentsByType.Select(new
{
name = grp.Key,
count = grp.Count(),
total = grp.Sum(x => x.pay.Amount)
}
);
Now that you have composed the query, execute it by enumerating. No DB access has happened until now.
var output = result.ToArray(); // <- DB access happens here
Edit After the suggestion of Ivan, I looked at our codebase. As he mentioned, the Extension methods should work on IQueryable instead of IEnumerable. Just take care that you only use expressions that can be translated to SQL, i.e. do not call any custom code like an overriden ToString() method.
Edit 2 If Payment and other model classes inherit BaseTable, the filter methods can be written as generic methods that accept any child type of BaseTable. Also added example for grouping method.
Currently I have an entity that is "geolocatable" via a SqlGeography column that I can use via expressions for filtering and sorting. I am already able to get all entities within distance x of point y and sort by entities closest to (or furthest from) point y. However, in order to return the distance from the entity to y I have to recalculate the distance in the application because I have not yet determined how to materialize the result of the distance calculation from the database to the entities in the IQueryable. This is a mapped entity and a great deal of application logic surrounds the type of entity returned so projecting it into a dynamic object is not a viable option for this implementation (though I understand how that would work). I have also tried using an unmapped object that inherits from the mapped entity but that suffers the same problems. Essentially, as I understand it, I should be able to define the getter of an unmapped property to assign a computed value in a queryable extension IF I modify the expression tree that represents the IQueryable but the how escapes me. I've written expressions in this manner before but I think I need to be able to modify the existing select rather than just chaining on a new Expression.Call which is unexplored territory for me.
The following should code should properly illustrate the problem:
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
using System.Data.Entity.ModelConfiguration;
using System.Data.Entity.Spatial; // from Microsoft.SqlServer.Types (Spatial) NuGet package
using System.Linq;
public class LocatableFoo
{
[Key]
public int Id { get; set; }
public DbGeography Geolocation { get; set; }
[NotMapped]
public double? Distance { get; set; }
}
public class PseudoLocatableFoo : LocatableFoo
{
}
public class LocatableFooConfiguration : EntityTypeConfiguration<LocatableFoo>
{
public LocatableFooConfiguration()
{
this.Property(foo => foo.Id).HasColumnName("id");
this.Property(foo => foo.Geolocation).HasColumnName("geolocation");
}
}
public class ProblemContext : DbContext
{
public DbSet<LocatableFoo> LocatableFoos { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new LocatableFooConfiguration());
base.OnModelCreating(modelBuilder);
}
}
public class Controller
{
public Controller(ProblemContext context) // dependency injection
{
this.Context = context;
}
private ProblemContext Context { get; set; }
/* PROBLEM IN THIS METHOD:
* Do not materialize results (ie ToList) and then calculate distance as is done currently <- double calculation of distance in DB and App I am trying to solve
* Must occur prior to materialization
* Must be assignable to "query" that is to type IQueryable<LocatableFoo>
*/
public IEnumerable<LocatableFoo> GetFoos(decimal latitude, decimal longitude, double distanceLimit)
{
var point = DbGeography.FromText(string.Format("Point({0} {1})", longitude, latitude), 4326); // NOTE! This expects long, lat rather than lat, long.
var query = this.Context.LocatableFoos.AsQueryable();
// apply filtering and sorting as proof that EF can turn this into SQL
query = query.Where(foo => foo.Geolocation.Distance(point) < distanceLimit);
query = query.OrderBy(foo => foo.Geolocation.Distance(point));
//// this isn't allowed because EF doesn't allow projecting to mapped entity
//query = query.Select( foo => new LocatableFoo { Id = foo.Id, Geolocation = foo.Geolocation, Distance = foo.Geolocation.Distance(point) });
//// this isn't allowed because EF doesn't allow projecting to mapped entity and PseudoLocatableFoo is considered mapped since it inherits from LocatableFoo
//query = query.Select( foo => new PseudoLocatableFoo { Id = foo.Id, Geolocation = foo.Geolocation, Distance = foo.Geolocation.Distance(point) });
//// this isn't allowed because we must be able to continue to assign to query, type must remain IQueryable<LocatableFoo>
//query = query.Select( foo => new { Id = foo.Id, Geolocation = foo.Geolocation, Distance = foo.Geolocation.Distance(point) });
// this is what I though might work
query = query.SelectWithDistance(point);
this.Bar(query);
var results = query.ToList(); // run generated SQL
foreach (var result in results) //problematic duplicated calculation
{
result.Distance = result.Geolocation.Distance(point);
}
return results;
}
// fake method representing lots of app logic that relies on knowing the type of IQueryable<T>
private IQueryable<T> Bar<T>(IQueryable<T> foos)
{
if (typeof(T) == typeof(LocatableFoo))
{
return foos;
}
throw new ArgumentOutOfRangeException("foos");
}
}
public static class QueryableExtensions
{
public static IQueryable<T> SelectWithDistance<T>(this IQueryable<T> queryable, DbGeography pointToCalculateDistanceFrom)
{
/* WHAT DO?
* I'm pretty sure I could do some fanciness with Expression.Assign but I'm not sure
* What to get the entity with "distance" set
*/
return queryable;
}
}
What about replacing the line
var results = query.ToList();
with
var results = query
.Select(x => new {Item = x, Distance = x.Geolocation.Distance(point)}
.AsEnumerable() // now you just switch to app execution
.Select(x =>
{
x.Item.Distance = x.Distance; // you don't need to calculate, this should be cheap
return x.Item;
})
.ToList();
The Distance field is logically not part of your table, since it represents a distance to a dynamically specified point. As such it should not be part of your entity.
At this point if you want it being calculated on the db, you should create a Stored procedure, or a TVF (or sg else) that returns your entity extended with the distance. This way you can map the return type to an Entity. It is a clearer design to me btw.
I'm using ActiveRecord on Subsonic 3 and I effectively want to do this:
select * from foo
left outer join bar on bar.Id = foo.barId
where foo.someProperty = 2
I've written a stored procedure to fetch the data but Subsonic has only created objects to hold the columns from foo and bar.
What's the best way of returning the data into a single object so I can just bind it. Ideally I want it to be in a list<> but without writing my own class, there doesn't seem to be a way provided by subsonic.
You have a couple options here...
You could create a database view that does your join, and have SubSonic generate a data type for your view, then your select would be just like selecting from any other table.
Alternatively, you could use a Linq expression to do the join into an anonymous or dynamic type (if you are using .net 4) For example:
public List<dynamic> LoadData(int id)
{
var data = from f in db.Foo
from b in db.Bar.Where(x => x.Id == f.BarId).DefaultIfEmpty()
where f.SomeProperty == id
select new
{
SomeProperty = f.Something,
AnotherProperty = b.SomethingElse
};
return data.Cast<dynamic>().ToList();
}
Of course another alternative is to do the Linq expression above, but define your own class to hold the returned data, and select into it.
public class MyData
{
public string SomeProperty { get; set; }
public string AnotherProperty { get; set; }
}
public List<MyData> LoadData(int id)
{
var data = from f in db.Foo
from b in db.Bar.Where(x => x.Id == f.BarId).DefaultIfEmpty()
where f.SomeProperty == id
select new MyData()
{
SomeProperty = f.Something,
AnotherProperty = b.SomethingElse
};
return data.ToList();
}