Develop Reference

How do I obtain all the most recent elements from an IEnumerable?

I'm developing a Windows Forms application project for my university and we are using Entity Framework to store things.
It's an e-commerce type of program and now I'm struggling to find the right way to filter an IEnumerable based on the most recent ones.
What I want is to obtain all the elements from this table called prices, in which we also store older prices as a history backup.
This table has the ID of the article that refers to, the same for the corresponding prices list, a public, and a cost price, the updated date that is the moment it was created/updated.
I have tried using many expressions but ultimately failed miserably, sometimes I brought me only the ones within a certain price list or none at all or just one.
Again, I need it to work for a function that lets you update your prices based on parameters. For example, all articles and all price lists. For that, I want only the ones that are up to date so I won't touch the history of prices.
Example of what it should return:
Thank you very much!
Update: What I have tried didn't work, in fact, I couldn't even find sense in the code I wrote, that's why I didn't post it in the first place. I guess this problem ended my brain and I can't think properly anymore.
I tried some answers that I found here. For example:
// This is an IEnumerable of the price DTO class, which has the same properties as the table.
// It contains all the prices without a filter.
var prices= _priceService.Get();
// Attempt 1
var uptodatePrices= prices.GroupBy(x => x.ArticleId)
.Select(x => x.OrderByDescending(s => s.Date).FirstOrDefault());
// Attempt 2
uptodatePrices = prices.Select(x => new PriceDto
{
Date = prices.Where(z=> z.Id == x.Id).Max(g=>g.Date)
});

Ok, It sounds like you want to return the latest price for a combination of price list and article..
You're on the right path with your first attempt, but not quite there. The second attempt looks like pure frustration. :)
I believe the solution you will be looking for will be to group the products, then take the latest price for each group. To do that you need to use the values that identify your group as the group by expression, then sort the grouped results to take your desired one.
var uptodatePrices= prices.GroupBy(x => new { x.ArticloId, x.ListPrecioId} )
.Select(x => x.OrderByDescending(p => p.Date).First())
.ToList();
When you do a GroupBy, the value(s) you specify in the groupby expression become the "Key" of the result. The result also contains an IEnumerable representing the items from the original expression set (prices) that fit that group.
This selects the Price entity, you can change the Select to select a DTO/ViewModel to return, populated by the price instead as well.
In your case you were grouping by just the ArticloId, so you'd get back the latest entry for that Article, but not the combination of article and list price. In the above example I group by both article and list price, then tell it to Select from each group's set, take the latest Price record. I use First rather than FirstOrDefault as because I am grouping on combinations I know there will be at least 1 entry for each combination. (or else there would be no combination) Avoid using ...OrDefault unless you're sure, and are handling that no result may come back.

What you are working with are LINQ queries. If you only need to sort by most recent date, you can do that like this:
prices.OrderByDescending(price=>price.FechaActualizacion).ToList();
Make sure your Price model has the FechaActualizacion property.

How to improve this LINQ query to find users will all skills in list

I'm having some trouble optimizing this query.
This is to filter out users that have a particular set of skills.
These skills are sent to the server in the form of a list of their IDs, which are GUIDs.
unfortunately I cannot just get the user as easily as I would like because the last person working on this placed this in a SQL view.
This is where we try and find all users with all skills selected.
skillIDs is the list of GUIDs
Here is what it looks like
myview.Where(view => skillIDs.All(skill => view.User.Skills.Any(s => s.ID == skill)))
Other things we have tried
myView.Where(view => !skillIDs.Except(view.User.Skills.Select(skill => skill.ID).Any()))
myview.Where(view => skillIDs.All(skill => view.User.Skills.Select(s => s.ID).contains(skill)))
I realize the way it is working is highly inefficient and yes, we are paginating the results, but not until after this query. What I believe is happening is it is executing the query here rather than waiting for the .skip(0).Take(10).tolist() which is when it should execute. Right now it takes 45 seconds for this to work.When it's not trying to execute the query above it's less than a second so I know this is the culprit.

In this case playing with different variations of LINQ won't make a difference as the issue is likely in the backend table indexing, not in how you create a LINQ statement. You really have two options:
Index the backend table as the view will be able to use index on the table if it needs
Index a view directly. The definition of an indexed view must be deterministic. MSDN
CREATE UNIQUE CLUSTERED INDEX IDX_V1
ON myview (skillid);
GO

Linq Select many where property x exists in externalList

Poor wording I know, so I will be clear here..
I have
List<int> relevantIDs; //self explanitory
Now I am trying to select a list of objects from the database where their IDs exist within the list above... I cant seem to figure out the linq...
dbContext.objs.where(x => x.id ....).toList();
//I cant figure out the .... i was thinking there was an "in" but got no where...
can someone point me to an article or provide me a sample that will accomplish what I need. The list would be too big to just retrieve them all then filter down.... and hitting the database repeatedly would be not optimal in this case...

You want to find the IDs from the database that are contained within your collection of "relevant IDs", so this should work:
dbContext.objs.Where(x => relevantIDs.Contains(x.id)).ToList();

You are looking for Contains
dbContext.objs.Where(x => relevantIDs.Contains(x.id)).ToList();
This will be translated as IN clause into SQL.

Entity Framework COUNT is doing a SELECT of all records

Profiling my code because it is taking a long time to execute, it is generating a SELECT instead of a COUNT and as there are 20,000 records it is very very slow.
This is the code:
var catViewModel= new CatViewModel();
var catContext = new CatEntities();
var catAccount = catContext.Account.Single(c => c.AccountId == accountId);
catViewModel.NumberOfCats = catAccount.Cats.Count();
It is straightforward stuff, but the code that the profiler is showing is:
exec sp_executesql N'SELECT
[Extent1].xxxxx AS yyyyy,
[Extent1].xxxxx AS yyyyy,
[Extent1].xxxxx AS yyyyy,
[Extent1].xxxxx AS yyyyy // You get the idea
FROM [dbo].[Cats] AS [Extent1]
WHERE Cats.[AccountId] = #EntityKeyValue1',N'#EntityKeyValue1 int',#EntityKeyValue1=7
I've never seen this behaviour before, any ideas?
Edit: It is fixed if I simply do this instead:
catViewModel.NumberOfRecords = catContext.Cats.Where(c => c.AccountId == accountId).Count();
I'd still like to know why the former didn't work though.

So you have 2 completely separate queries going on here and I think I can explain why you get different results. Let's look at the first one
// pull a single account record
var catAccount = catContext.Account.Single(c => c.AccountId == accountId);
// count all the associated Cat records against said account
catViewModel.NumberOfCats = catAccount.Cats.Count();
Going on the assumption that Cats has a 0..* relationship with Account and assuming you are leveraging the frameworks ability to lazily load foreign tables then your first call to catAccounts.Cats is going to result in a SELECT for all the associated Cat records for that particular account. This results in the table being brought into memory therefore the call to Count() would result in an internal check of the Count property of the in-memory collection (hence no COUNT SQL generated).
The second query
catViewModel.NumberOfRecords =
catContext.Cats.Where(c => c.AccountId == accountId).Count();
Is directly against the Cats table (which would be IQueryable<T>) therefore the only operations performed against the table are Where/Count, and both of these will be evaluated on the DB-side before execution so it's obviously a lot more efficient than the first.
However, if you need both Account and Cats then I would recommend you eager load the data on the fetch, that way you take the hit upfront once
var catAccount = catContext.Account.Include(a => a.Cats).Single(...);

Most times, when somebody accesses a sub-collection of an entity, it is because there are a limited number of records, and it is acceptable to populate the collection. Thus, when you access:
catAccount.Cats
(regardless of what you do next), it is filling that collection. Your .Count() is then operating on the local in-memory collection. The problem is that you don't want that. Now you have two options:
check whether your provider offer some mechanism to make that a query rather than a collection
build the query dynamically
access the core data-model instead
I'm pretty confident that if you did:
catViewModel.NumberOfRecords =
catContext.Cats.Count(c => c.AccountId == accountId);
it will work just fine. Less convenient? Sure. But "works" is better than "convenient".

how to append IQueryable within a loop

I have a simple foreach loop that goes through the productID's I have stored in a user's basket and looks up the product's details from the database.
As you can see from my code, what I have at present will return the very last item on screen - as the variable is overwritten within the loop. I'd like to be able to concat this so that I can display the product details for the items only in the basket.
I know I could do something very easy like store only ProductIDs in the repeater I use and onitemdatabound call the database there but I'd like to make just one database call if possible.
Currently I have the following (removed complex joins from example, but if this matters let me know):
IQueryable productsInBasket = null;
foreach (var thisproduct in store.BasketItems)
{
productsInBasket = (from p in db.Products
where p.Active == true && p.ProductID == thisproduct.ProductID
select new
{
p.ProductID,
p.ProductName,
p.BriefDescription,
p.Details,
p.ProductCode,
p.Barcode,
p.Price
});
}
BasketItems.DataSource = productsInBasket;
BasketItems.DataBind();
Thanks for your help!

It sounds like you really want something like:
var productIds = store.BasketItems.Select(x => x.ProductID).ToList();
var query = from p in db.Products
where p.Active && productIds.Contains(p.ProductID)
select new
{
p.ProductID,
p.ProductName,
p.BriefDescription,
p.Details,
p.ProductCode,
p.Barcode,
p.Price
};

In Jon's answer, which works just fine, the IQueryable will however be converted to an IEnumerable, since you call ToList() on it. This will cause the query to be executed and the answer retrieved. For your situation, this may be OK, since you want to retrieve products for a basket, and where the number of products will probably be considerably small.
I am, however, facing a similar situation, where I want to retrieve friends for a member. Friendship depends on which group two members belongs to - if they share at least one group, they are friends. I thus have to retrieve all membership for all groups for a certain member, then retrieve all members from those groups.
The ToList-approach will not be applicable in my case, since that would execute the query each time I want to handle my friends in various ways, e.g. find stuff that we can share. Retrieving all members from the database, instead of just working on the query and execute it at the last possible time, will kill performance.
Still, my first attempt at this situation was to do just this - retrieve all groups I belonged to (IQueryable), init an List result (IEnumerable), then loop over all groups and append all members to the result if they were not already in the list. Finally, since my interface enforced that an IQueryable was to be returned, I returned the list with AsIQueryable.
This was a nasty piece of code, but at least it worked. It looked something like this:
var result = new List<Member>();
foreach (var group in GetGroupsForMember(member))
result.AddRange(group.GroupMembers.Where(x => x.MemberId != member.Id && !result.Contains(x.Member)).Select(groupMember => groupMember.Member));
return result.AsQueryable();
However, this is BAD, since I add ALL shared members to a list, then convert the list to an IQueryable just to satisfy my post condition. I will retrieve all members that are affected from the database, every time I want to do stuff with them.
Imagine a paginated list - I would then just want to pick out a certain range from this list. If this is done with an IQueryable, the query is just completed with a pagination statement. If this is done with an IEnumerable, the query has already been executed and all operations are applied to the in-memory result.
(As you may also notice, I also navigate down the entity's relations (GroupMember => Member), which increases coupling can cause all kinds of nasty situations further on. I wanted to remove this behavior as well).
So, tonight, I took another round and ended up with a much simpler approach, where I select data like this:
var groups = GetGroupsForMember(member);
var groupMembers = GetGroupMembersForGroups(groups);
var memberIds = groupMembers.Select(x => x.MemberId);
var members = memberService.GetMembers(memberIds);
The two Get methods honor the IQueryable and never convert it to a list or any other IEnumerable. The third line just performs a LINQ query ontop of the IEnumerable. The last line just takes the member IDs and retrieves all members from another service, which also works exclusively with IQueryables.
This is probably still horrible in terms of performance, but I can optimize it further later on, if needed. At least, I avoid loading unnecessary data.
Let me know if I am terribly wrong here.