I am trying to cut this linq down
var sys = db.tlkpSystems
.Where(a => db.tlkpSettings.Where(e => e.Hidden < 3)
.Select(o => o.System)
.ToList().Contains(a.System)) //cannot get this part in?
.OrderBy(a => a.SystemName).ToList();
foreach (var item in sys)
model.Add(new SettingSystem {
System = item.System,
SystemName = item.SystemName
});
I have tried the following:
List<SettingSystem> model = new List<SettingSystem>();
model = db.tlkpSettings.Where(e => e.Hidden < 3)
.OrderBy(e => e.Setting)
.Select(e => new SettingSystem
{
System = e.System,
SystemName = e.Setting
}).ToList();
How can I call the .Contains(a.System) part in my query?
Thanks
Some general rules when working with LINQ to Entities:
Avoid using ToList inside the query. It prevents EF to build a correct SQL query.
Don't use Contains when working with entities (tables). Use Any or joins.
Here is your query (in case System is not an entity navigation property):
var sys = db.tlkpSystems
.Where(a => db.tlkpSettings.Any(e => e.Hidden < 3 && e.System == a.System))
.OrderBy(a => a.SystemName).ToList();
As an addendum, there is also AsEnumerable for when you must pull a query into memory (such as calling methods within another clause). This is generally better than ToList or ToArray since it'll enumerate the query, rather than enumerating, putting together a List/Array, and then enumerating that collection.
Related
I'm looking for suggestions on how to write a query. For each Goal, I want to select the first Task (sorted by Task.Sequence), in addition to any tasks with ShowAlways == true. (My actual query is more complex, but this query demonstrates the limitations I'm running into.)
I tried something like this:
var tasks = (from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
let nextTaskId = g.Tasks.OrderBy(tt => tt.Sequence).Select(tt => tt.Id).DefaultIfEmpty(-1).FirstOrDefault()
where t.ShowAlways || t.Id == nextTaskId
select new CalendarTask
{
// Member assignment
}).ToList();
But this query appears to be too complex.
System.InvalidOperationException: 'Processing of the LINQ expression 'OrderBy<Task, int>(
source: MaterializeCollectionNavigation(Navigation: Goal.Tasks(< Tasks > k__BackingField, DbSet<Task>) Collection ToDependent Task Inverse: Goal, Where<Task>(
source: NavigationExpansionExpression
Source: Where<Task>(
source: DbSet<Task>,
predicate: (t0) => Property<Nullable<int>>((Unhandled parameter: ti0).Outer.Inner, "Id") == Property<Nullable<int>>(t0, "GoalId"))
PendingSelector: (t0) => NavigationTreeExpression
Value: EntityReferenceTask
Expression: t0
,
predicate: (i) => Property<Nullable<int>>(NavigationTreeExpression
Value: EntityReferenceGoal
Expression: (Unhandled parameter: ti0).Outer.Inner, "Id") == Property<Nullable<int>>(i, "GoalId"))),
keySelector: (tt) => tt.Sequence)' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.'
The problem is the line let nextTaskId =.... If I comment out that, there is no error. (But I don't get what I'm after.)
I'll readily admit that I don't understand the details of the error message. About the only other way I can think of to approach this is return all the Tasks and then sort and filter them on the client. But my preference is not to retrieve data I don't need.
Can anyone see any other ways to approach this query?
Note: I'm using the very latest version of Visual Studio and .NET.
UPDATE:
I tried a different, but less efficient approach to this query.
var tasks = (DbContext.Areas
.Where(a => a.UserId == UserManager.GetUserId(User) && !a.OnHold)
.SelectMany(a => a.Goals)
.Where(g => !g.OnHold)
.Select(g => g.Tasks.Where(tt => !tt.OnHold && !tt.Completed).OrderBy(tt => tt.Sequence).FirstOrDefault()))
.Union(DbContext.Areas
.Where(a => a.UserId == UserManager.GetUserId(User) && !a.OnHold)
.SelectMany(a => a.Goals)
.Where(g => !g.OnHold)
.Select(g => g.Tasks.Where(tt => !tt.OnHold && !tt.Completed && (tt.DueDate.HasValue || tt.AlwaysShow)).OrderBy(tt => tt.Sequence).FirstOrDefault()))
.Distinct()
.Select(t => new CalendarTask
{
Id = t.Id,
Title = t.Title,
Goal = t.Goal.Title,
CssClass = t.Goal.Area.CssClass,
DueDate = t.DueDate,
Completed = t.Completed
});
But this also produced an error:
System.InvalidOperationException: 'Processing of the LINQ expression 'Where<Task>(
source: MaterializeCollectionNavigation(Navigation: Goal.Tasks (<Tasks>k__BackingField, DbSet<Task>) Collection ToDependent Task Inverse: Goal, Where<Task>(
source: NavigationExpansionExpression
Source: Where<Task>(
source: DbSet<Task>,
predicate: (t) => Property<Nullable<int>>((Unhandled parameter: ti).Inner, "Id") == Property<Nullable<int>>(t, "GoalId"))
PendingSelector: (t) => NavigationTreeExpression
Value: EntityReferenceTask
Expression: t
,
predicate: (i) => Property<Nullable<int>>(NavigationTreeExpression
Value: EntityReferenceGoal
Expression: (Unhandled parameter: ti).Inner, "Id") == Property<Nullable<int>>(i, "GoalId"))),
predicate: (tt) => !(tt.OnHold) && !(tt.Completed))' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.'
This is a good example for the need of full reproducible example. When trying to reproduce the issue with similar entity models, I was either getting a different error about DefaulIfEmpty(-1) (apparently not supported, don't forget to remove it - the SQL query will work correctly w/o it) or no error when removing it.
Then I noticed a small deeply hidden difference in your error messages compared to mine, which led me to the cause of the problem:
MaterializeCollectionNavigation(Navigation: Goal.Tasks (<Tasks>k__BackingField, DbSet<Task>)
specifically the DbSet<Task> at the end (in my case it was ICollection<Task>). I realized that you used DbSet<T> type for collection navigation property rather than the usual ICollection<T>, IEnumerable<T>, List<T> etc., e.g.
public class Goal
{
// ...
public DbSet<Task> Tasks { get; set; }
}
Simply don't do that. DbSet<T> is a special EF Core class, supposed to be used only from DbContext to represent db table, view or raw SQL query result set. And more importantly, DbSets are the only real EF Core query roots, so it's not surprising that such usage confuses the EF Core query translator.
So change it to some of the supported interfaces/classes (for instance, ICollection<Task>) and the original problem will be solved.
Then removing the DefaultIfEmpty(-1) will allow successfully translating the first query in question.
I don't have EF Core up and running, but are you able to split it up like this?
var allTasks = DbContext.Areas
.SelectMany(a => a.Goals)
.SelectMany(a => a.Tasks);
var always = allTasks.Where(t => t.ShowAlways);
var next = allTasks
.OrderBy(tt => tt.Sequence)
.Take(1);
var result = always
.Concat(next)
.Select(t => new
{
// Member assignment
})
.ToList();
Edit: Sorry, I'm not great with query syntax, maybe this does what you need?
var allGoals = DbContext.Areas
.SelectMany(a => a.Goals);
var allTasks = DbContext.Areas
.SelectMany(a => a.Goals)
.SelectMany(a => a.Tasks);
var always = allGoals
.SelectMany(a => a.Tasks)
.Where(t => t.ShowAlways);
var nextTasks = allGoals
.SelectMany(g => g.Tasks.OrderBy(tt => tt.Sequence).Take(1));
var result = always
.Concat(nextTasks)
.Select(t => new
{
// Member assignment
})
.ToList();
I would recommend you start by breaking up this query into individual parts. Try iterating through the Goals in a foreach with your Task logic inside. Add each new CalendarTask to a List that you defined ahead of time.
Overall breaking this logic up and experimenting a bit will probably lead you to some insight with the limitations of Entity Framework Core.
I think we might separate the query into two steps. First, query each goals and get the min Sequence task and store them(maybe with a anonymous type like {NextTaskId,Goal}). Then, we query the temp data and get the result. For example
Areas.SelectMany(x=>x.Goals)
.Select(g=>new {
NextTaskId=g.Tasks.OrderBy(t=>t.Sequence).FirstOrDefault()?.Id,
Tasks=g.Tasks.Where(t=>t.ShowAlways)
})
.SelectMany(a=>a.Tasks,(a,task)=>new {
NextTaskId = a.NextTaskId,
Task = task
});
I tried to create the linq request but I'm not sure about the result
var tasks = ( from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
join oneTask in (from t in DbContext.Tasks
group t by t.Id into gt
select new {
Id = gt.Key,
Sequence = gt.Min(t => t.Sequence)
}) on new { t.Id, t.Sequence } equals new { oneTask.Id,oneTask.Sequence }
select new {Area = a, Goal = g, Task = t})
.Union(
from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
where t.ShowAlways
select new {Area = a, Goal = g, Task = t});
I currently don't have EF Core, but do you really need to compare this much?
Wouldn't querying the tasks be sufficient?
If there is a navigation property or foreign key defined I could imaging using something like this:
Tasks.Where(task => task.Sequence == Tasks.Where(t => t.GoalIdentity == task.GoalIdentity).Min(t => t.Sequence) || task.ShowAlways);
I am trying to filter out the second part of the tables (UserRoles.IsDeleted==false). Is there any advice how i can do that?
var Users = context.Users.Where(r => r.IsDeleted == IsDeleted).ToList<User>();
Users = context.Users.Include(x => x.UserRoles.Select(y=>y.IsDeleted==false)).ToList();
Thank you
You can do the following to filter using the second part:
var Users = context.Users.Where(r => r.IsDeleted == IsDeleted).ToList<User>();
if(condition)
{
Users = Users.where(y => y.IsDeleted == false)).ToList();
}
There are two options to filter related entities
Doing a projection.
Unfortunately, when you use Include method, you can't filter the related entities as you intend to do. You need to project your query to a DTO object or a anonymous object, as the below example.
var query=context.Users.Include(x => x.UserRoles)
.Where(r => r.IsDeleted == IsDeleted)
.Select(u=> new{ ...,
Roles=x => x.UserRoles.Where(y=>!y.IsDeleted)})
A second option could be using Explicitly Loading. But this is in case you can load the related entities of one specific entity,eg,.
var user=context.Users.FirstOrDefault(r.IsDeleted == IsDeleted);//Getting a user
context.Entry(user)
.Collection(b => b.UserRoles)
.Query()
.Where(y=>!y.IsDeleted)
.Load();
You can do this inside of a foreach per each entity you get from the first query,
var query=context.Users.Where(r => r.IsDeleted == IsDeleted);
foreach(var u in query)
{
context.Entry(u)
.Collection(b => b.UserRoles)
.Query()
.Where(y=>!y.IsDeleted)
.Load();
}
but it's going to be really inefficient because you are going to do a roundtrip to your DB per each entity. My advice is use the first option, projecting the query.
I have two options to filter my query:
var users = query.Select(x => x.User).FindUsers(searchString);
query = query.Where(x => users.Contains(x.User));
or
var userIds = query.Select(x => x.User).FindUsers(searchString).Select(x => x.Id);
query = query.Where(x => userIds.Contains(x.UserId));
Description of FindUsers extension:
static IQueryable<User> FindUsers(this IQueryable<User> query, string searchString);
So what I will have in SQL finally? Which of these two requests are better for perfomance?
If someone has another suggestion write it in answer please.
Thanks in advance!
Both query are similar in EF v6 or higher; Contains condition will be translated to Sql EXISTS
Take a loot to the code below :
using (var dbContext = new DbContextTest("DatabaseConnectionString"))
{
var users = dbContext.Users.Select(u => u).Where(x => x.UserId > 0);
var query = dbContext.Users.Where(x => users.Contains(x));
Console.WriteLine(query.ToList());
}
using (var dbContext = new DbContextTest("DatabaseConnectionString"))
{
var ids = dbContext.Users.Select(u => u.UserId).Where(x => x > 0);
var query = dbContext.Users.Where(x => ids.Contains(x.UserId));
Console.WriteLine(query.ToList());
}
The output Sql queries are excatly the same (you can use profiler or EF logger to see that). One thing maybe is important, selecting only the Id's will be more agile for materialisation and cache.
Tip:
If you add ToList() in the Ids query dbContext.Users.Select(u => u.UserId).Where(x => x > 0).ToList(); then this fix will imporve your result performance. The next select query will be translated with SQL "IN" instead of of "EXISTS"! Take a look to this link Difference between EXISTS and IN in SQL? and you can decide what is better for you.
Note:
ToList() will materialize the Id's, that means you will work with another interface then IQueryable!
this code written by #Rahul Singh in this post Convert TSQL to Linq to Entities :
var result = _dbContext.ExtensionsCategories.ToList().GroupBy(x => x.Category)
.Select(x =>
{
var files = _dbContext.FileLists.Count(f => x.Select(z => z.Extension).Contains(f.Extension));
return new
{
Category = x.Key,
TotalFileCount = files
};
});
but this code have problem when used inside database context and we should use ToList() like this to fix "Only primitive types or enumeration types are supported in this context" error :
var files = _dbContext.FileLists.Count(f => x.Select(z => z.Extension).ToList().Contains(f.Extension));
the problem of this is ToList() fetch all records and reduce performance, now i wrote my own code :
var categoriesByExtensionFileCount =
_dbContext.ExtensionsCategories.Select(
ec =>
new
{
Category = ec.Category,
TotalSize = _dbContext.FileLists.Count(w => w.Extension == ec.Extension)
});
var categoriesTOtalFileCount =
categoriesByExtensionFileCount.Select(
se =>
new
{
se.Category,
TotalCount =
categoriesByExtensionFileCount.Where(w => w.Category == se.Category).Sum(su => su.TotalSize)
}).GroupBy(x => x.Category).Select(y => y.FirstOrDefault());
the performance of this code is better but it have much line of code, any idea about improve performance of first code or reduce line of second code :D
Regards, Mojtaba
You should have a navigation property from ExtensionCategories to FileLists. If you are using DB First, and have your foreign key constraints set up in the database, it should do this automatically for you.
If you supply your table designs (or model classes), it would help a lot too.
Lastly, you can rewrite using .ToList().Contains(...) with .Any() which should solve your immediate issue. Something like:
_dbContext.FileLists.Count(f => x.Any(z => z.Extension==f.Extension)));
I have a method that returns data from an EF model.
I'm getting the above message, but I can't wotk our how to circumvent the problem.
public static IEnumerable<FundedCount> GetFundedCount()
{
var today = DateTime.Now;
var daysInMonth = DateTime.DaysInMonth(today.Year, today.Month);
var day1 = DateTime.Now.AddDays(-1);
var day31 = DateTime.Now.AddDays(-31);
using (var uow = new UnitOfWork(ConnectionString.PaydayLenders))
{
var r = new Repository<MatchHistory>(uow.Context);
return r.Find()
.Where(x =>
x.AppliedOn >= day1 && x.AppliedOn <= day31 &&
x.ResultTypeId == (int)MatchResultType.Accepted)
.GroupBy(x => new { x.BuyerId, x.AppliedOn })
.Select(x => new FundedCount(
x.Key.BuyerId,
x.Count() / 30 * daysInMonth))
.ToList();
}
}
FundedCount is not an EF enity, MatchHistory is, so can't understand why it is complaining.
All advice appreciated.
The reason it is complaining is because it doesn't know how to translate your Select() into a SQL expression. If you need to do a data transformation to a POCO that is not an entity, you should first get the relevant data from EF and then transform it to the POCO.
In your case it should be as simple as calling ToList() earlier:
return r.Find()
.Where(x => x.AppliedOn >= day1 && x.AppliedOn <= day31 &&
x.ResultTypeId == (int)MatchResultType.Accepted)
.GroupBy(x => new { x.BuyerId, x.AppliedOn })
.ToList() // this causes the query to execute
.Select(x => new FundedCount(x.Key.BuyerId, x.Count() / 30 * daysInMonth));
Be careful with this, though, and make sure that you're limiting the size of the data set returned by ToList() as much as possible so that you're not trying to load an entire table into memory.
Message is clear : linq to entities doesn't support objects without a parameterless ctor.
So
Solution1
enumerate before (or use an intermediate anonymous type and enumerate on that one)
.ToList()
.Select(x => new FundedCount(
x.Key.BuyerId,
x.Count() / 30 * daysInMonth))
.ToList();
Solution2
add a parameterless ctor to your FundedCount class (if it's possible)
public FundedCount() {}
and use
.Select(x => new FundedCount{
<Property1> = x.Key.BuyerId,
<Property2> = x.Count() / 30 * daysInMonth
})
.ToList();
It's complaining because it can't convert references to FundedCount to SQL statements.
All LINQ providers convert LINQ statements and expressions to operations that their target can understand. LINQ to SQL and LINQ to EF will convert LINQ to SQL, PLINQ will convert it to Tasks and parallel operations, LINQ to Sharepoint will convert it to CAML etc.
What happens if they can't do the conversion, depends on the provider. Some providers will return intermediate results and convert the rest of the query to a LINQ to Objects query. Others will simply fail with an error message.
Failing with a message is actually a better option when talking to a database. Otherwise the server would have to return all columns to the client when only 1 or 2 would be actually necessary.
In your case you should modify your select to return an anonymous type with the data you want, call ToList() and THEN create the FundedCount objects, eg:
.Select( x=> new {Id=x.Key.BuyerId,Count=x.Count()/30 * daysInMonth)
.ToList()
.Select(y => new FundedCount(y.Id,y.Count))
.ToList();
The first ToList() will force the generation of the SQL statement and execute the query that will return only the data you need. The rest of the query is actually Linq to Objects and will get the data and create the final objects
I had the same exception in GroupBy. I found that the exception "Only parameterless constructors and initializers are supported in LINQ to Entities" is not 100% accurate description.
I had a GroupBy() in my "Linq to EntityFramework query" which used a struct as a Key in GroupBy. That did not work. When I changed that struct to normal class everything worked fine.
Code sample
var affectedRegistrationsGrouped = await db.Registrations
.Include(r => r.Person)
.Where(r =>
//whatever
)
.GroupBy(r => new GroupByKey
{
EventId = r.EventId,
SportId = r.SportId.Value
})
.ToListAsync();
...
...
// this does not work
private struct GroupByKey() {...}
// this works fine
private class GroupByKey() {...}