Is nested GroupBy() then flatten allowed in SQL? or available via ling-to-sql or the entity framework? Currently, I need to perform a retrieve in the middle of the query to make it work:
var query = (
from s in Prices
group s by new { s.P1, s.P2 } into FirstGroups
select FirstGroups
)
.ToList() // without it, exception is thrown
.SelectMany(g1 =>
g1.GroupBy(i => i.P3).OrderBy(i => i.Key).Take(2)
.SelectMany((g2, index) => g.Select(j => new
{
P1 = g1.Key.P1,
P2 = g1.Key.P2,
Index = index,
P3 = g2.P3,
P4 = j.P4,
}));
});
Single SelectMany works. Nested expressed this way doesn't work in linq-to-sql. My question is does l2s support it at all? if yes, how to write the query. if not, does any other linq to db technology support it, for example, the new entity framework?
Grouping and then flattening the results is the same as simply ordering on that value.
Related
The challenge is about converting from method chain to standard linq a piece of code full of group by.
The context
To fully understand the topic here you can read the original question (with class definitions, sample data and so on): Linq: rebuild hierarchical data from the flattened list
Thanks to #Akash Kava, I've found the solution to my problem.
Chain method formulation
var macroTabs = flattenedList
.GroupBy(x => x.IDMacroTab)
.Select((x) => new MacroTab
{
IDMacroTab = x.Key,
Tabs = x.GroupBy(t => t.IDTab)
.Select(tx => new Tab {
IDTab = tx.Key,
Slots = tx.Select(s => new Slot {
IDSlot = s.IDSlot
}).ToList()
}).ToList()
}).ToList();
But, for sake of knowledge, I've tried to convert the method chain to the standard Linq formulation but something is wrong.
What happens is similar to this..
My attempt to convert it to Linq standard syntax
var antiflatten = flattenedList
.GroupBy(x => x.IDMacroTab)
.Select(grouping => new MacroTab
{
IDMacroTab = grouping.Key,
Tabs = (from t in grouping
group grouping by t.IDTab
into group_tx
select new Tab
{
IDTab = group_tx.Key,
Slots = (from s in group_tx
from s1 in s
select new Slot
{
IDSlot = s1.IDSlot
}).ToList()
}).ToList()
});
The result in LinqPad
The classes and the sample data on NetFiddle:
https://dotnetfiddle.net/8mF1qI
This challenge helped me to understand what exactly returns a Linq Group By (and how prolix is the Linq syntax with Group By).
As LinqPad clearly shows a Group By returns a List of Groups. Group is a very simple class which has just one property: a Key
As this answer states, from definition of IGrouping (IGrouping<out TKey, out TElement> : IEnumerable<TElement>, IEnumerable) the only way to access to the content of the subgroups is to iterate through elements (a foreach, another group by, a select, ecc).
Here is shown the Linq syntax formulation of the method chain.
And here is the source code on Fiddle
But let's go on trying to see another solution:
What we usually do in SQL when we do a Group By is to list all the columns but the one which have been grouped. With Linq is different.. it still returns ALL the columns.
In this example we started with a dataset with 3 'columns' {IDMacroTab, IDTab, IDSlot}. We grouped for the first column, but Linq would return the whole dataset, unless we explicitly tell him..
I have a simple scenario.I want to list out all the employees except the logged in user.
Similar SQL Condition is
select * from employee where id not in(_loggedUserId)
How can I acheive the above using LINQ.I have tried the following query but not getting the desired list
int _loggedUserId = Convert.ToInt32(Session["LoggedUserId"]);
List<int> _empIds = _cmn.GetEmployeeCenterWise(_loggedUserId)
.Select(e => e.Id)
.Except(_loggedUserId)
.ToList();
Except expects argument of type IEnumerable<T>, not T, so it should be something like
_empIds = _cmn.GetEmployeeCenterWise(_loggedUserId)
.Select(e => e.Id)
.Except(new[] {_loggedUserId})
.ToList();
Also note, this is really redundant in the case when exclusion list contains only one item and can be replaces with something like .Where(x => x != _loggedUserId)
Why not use a very simple Where condition?
_empIds = _cmn.GetEmployeeCenterWise(_loggedUserId).Where(e=>e.Id != _loggedUserId).ToList();
The title of your question is how to perform a not in query against a database using LINQ. However, as others have pointed out your specific problem is better solved by a using users.Where(user => user.Id != loggedInUserId).
But there is still an answer on how to perform a query against a database using LINQ that results in NOT IN SQL being generated:
var userIdsToFilter = new[] { ... };
var filteredUsers = users.Where(user => !userIdsToFilter.Contains(user.Id));
That should generate the desired SQL using either Entity Framework or LINQ to SQL.
Entity Framework also allows you to use Except but then you will have to project the sequence to ID's before filtering them and if you need to original rows you need to fetch them again from the filtered sequence of ID's. So my advice is use Where with a Contains in the predicate.
Use LINQ without filtering. This will make your query execute much faster:
List<int> _empIds = _cmn.GetEmployeeCenterWise(_loggedUserId)
.Select(e => e.Id).ToList();
Now use List.Remove() to remove the logged-in user.
_empIds.Remove(_loggedUserId);
I have tried This answer, This one and this one to merge two iqueryables. But I always receive the following error:
The type 'Estudio' appears in two structurally incompatible initializations within a single LINQ to Entities query. A type can be initialized in two places in the same query, but only if the same properties are set in both places and those properties are set in the same order.
I'm mapping from two different but similar Entity Framework Entities (EXAMEN and EXPLORACION) to my domain entity Estudio, with the following code.
IQueryable<Estudio> listExamen = context.Set<EXAMEN>().Project().To<Estudio>();
IQueryable<Estudio> listExploracion = context.Set<EXPLORACION>().Project().To<Estudio>();
var listCombined = listExamen.Concat(listExploracion);
Is there anyway of generate a IQueryable (not enumerable) with the merging of both list? If AsEnumerable() is used, then the following filters (Order, Take, etc) are executed on memory. So I need to merge the list but still be able to apply filter to the merged list wihtout execute the queries.
//This will force the next condition is executed on memory
var listCombined = listExamen.AsEnumerable().Concat(listExploracion);
Is that possible?
I would try to select your data into an anonymous type in your linq query, perform the union, and add your criteria.
var listExamen = context.Examen
.Select(x => new { x.Prop1, x.Prop2, ... }); // Add properties
var listExploracion = context.Exploraction
.Select(x => new { x.Prop1, x.Prop2, ... }); // Add identical properties
var listCombined = listExamen.Concat(listExploracion);
var whereAdded = listCombines
.Where(x => x.Prop1 == someValue);
var result = whereAdded
.Skip(skipCount)
.Take(takeCount)
.ToList();
Note: I have no idea if you can use Common Table Expressions (the SQL necessity for skip/take) in combination with a Union-query
Note: I've changed the methods used to create the expressions, since I do not know your methods (Project, To)
So I think the solution is not to cast to a specific type, but to an anonymous type, since that probably can be translated to SQL.
Warning: didn't test it
My solution was to revise my mapping code. Instead of using individual property-based mappers, I had to project the entire entity at once, making sure that all of the properties were given in the same order.
So, instead of the ForMember syntax:
Mapper.CreateMap<Client, PersonResult>()
.ForMember(p => p.Name, cfg => cfg.MapFrom(c => c.Person.FirstName + " " + c.Person.LastName))
...
I used the ProjectUsing syntax:
Mapper.CreateMap<Client, PersonResult>()
.ProjectUsing(c => new PersonResult()
{
Name = c.Person.FirstName + " " + c.Person.LastName
...
});
This must be because of the way AutoMapper constructs its projections.
One way to work around this is to add dummy types:
class Estudio<T> : Estudio { }
And new mapping:
Mapper.CreateMap<Estudio , Estudio>();
Mapper.CreateMap<EXAMEN , Estudio<EXAMEN>>();
Mapper.CreateMap<EXPLORACION, Estudio<EXPLORACION>>();
One caveat is that all fields in Estudio need some value in mapping.
You can't use ignore. Returning 0 or "" is fine.
Now we can do:
var a = context.Set<EXAMEN>().ProjectTo<Estudio<EXAMEN>>();
var b = context.Set<EXPLORACION>().ProjectTo<Estudio<EXPLORACION>>();
return a.ProjectTo<Estudio>().Concat(b.ProjectTo<Estudio>());
I want to run this LINQ simple code to have record number in LINQ but result is beneath error
var model = _db2.Persons.Select(
(x, index) => new
{
rn = index + 1,
col1 = x.Id
}).ToList();
Error:
LINQ to Entities does not recognize the method
'System.Linq.IQueryable1[<>f__AnonymousType22
[System.Int32,System.Int32]] Select[Person,<>f__AnonymousType22](System.Linq.IQueryable1
[MvcApplication27.Models.Person], System.Linq.Expressions.Expression1[System.Func3
[MvcApplication27.Models.Person,System.Int32,<>f__AnonymousType2`2
[System.Int32,System.Int32]]])' method, and this method cannot be translated into a store
expression.
The problem is that LINQ to Entities doesn't understand how to convert that Select overload (the one that gives you the index) into a SQL query. You can fix this by first selecting the portion from the DB you need (to avoid selecting every column unnecessarily), then doing AsEnumerable() to take it as an IEnumerable<T> instead of an IQueryable<T>, and then doing the Select purely in C# (in short, IQueryable<T>s are converted to SQL, while IEnumerable<T>s are run in code).
var model = _db2.Persons.Select(x => x.Id).AsEnumerable().Select(
(id, index) => new
{
rn = index + 1,
col1 = id
}).ToList();
Note that the query as you have it appears to be unordered, so the id/index pairings can change each time you call this. If you expected consistency, you should order by something (e.g. _db2.Persons.OrderBy(...)).
Edit
Adding comment from Scott:
As a nice reference here is the list of all Linq statements built in
to the framework and a listing if it is compatible or not.
You could just select the Id and after it create your own anonymous object using linq to objects, for sample:
var model = _db2.Persons.Select(x => x.Id)
.ToList() // return int[]
.Select((id, index) => new
{
rn = index + 1,
col1 = id
}) // return anonymous[] (with rn and col1)
.AsEnumerable(); // get an IEnumerable (readonly collection)
Problably this is happen because Entity Framework does not support this kind of query using linq as linq could do in memory, so, in this case, you could select just you need (id in your case) and execute it, using ToList() method to concretize your query and after that you will have a list on memory, so, you can use linq to objects and use the supported method as you want.
Is there any mechanism for doing a JOIN between an in-memory collection and entity framework while preserving the order.
What I am trying is
var itemsToAdd =
myInMemoryList.Join(efRepo.All(), listitem => listitem.RECORD_NUMBER,
efRepoItem => efRepoItem.RECORD_NUMBER, (left, right) => right);
which gives me the rather curiously titled "This method supports the LINQ to Entities infrastructure and is not intended to be used directly from your code." error.
Now of course I can do this iteratively with something like
foreach (var item in myInMemoryList)
{
var ho = efRepo.Where(h => h.RECORD_NUMBER == item.RECORD_NUMBER).FirstOrDefault();
tmp.Add(ho);
}
but this is an N+1 query. Which is nasty as myInMemoryList might be quite large!
Resharper can refactor that for me to
tmp = (from TypeOfItemInTheList item in myInMemoryList
select efRepo.Where(h => h.RECORD_NUMBER == item.RECORD_NUMBER)
.FirstOrDefault());
which I suspect is still doing N+1 queries. So any ideas for a better approach to getting ef entities that match (on key field) with an in-memory collection. The resulting set must be in the same order as the in-memory collection was.
No you cannot join in-memory collection with database result set without loading whole result set to the memory and performing the join with linq-to-objects. Try using contains instead of join:
var myNumbers = myInMemoryList.Select(i => i.RECORD_NUMBER);
var itemsToAdd = efRepo.Where(e => myNumbers.Contains(e.RECORD_NUMBER));
This will generate query with IN operator
You can read how you can do this with the PredicateBuilder from the LINQKit or Stored Procedures in my blog post.
http://kalcik.net/2014/01/05/joining-data-in-memory-with-data-in-database-table/
try this:
var list = (from n in efRepo
where myInMemoryList.Select(m=>m.RECORD_NUMBER).Contains(n.RECORD_NUMBER)
select n).ToList();
Contains will be translated to IN operator in SQL (only if your RECORD_NUMBER member is a primitive type like int, string, Guid, etc)
What about loading the whole efRepo? I mean something like this (ToArray()):
var itemsToAdd = myInMemoryList.Join(
efRepo.ToArray(),
listitem => listitem.RECORD_NUMBER, efRepoItem => efRepoItem.RECORD_NUMBER, (left, right) => right);