I have a method which used for getting a list from the database.
public List<SelectedCustomers> GetCustomers(List<int> customerNumbers)
{
var customers=_context.Customers.Where(?).Select(i=> new SelectedCustomers() {}).ToList()
}
I want to retrieve information from the database of customers whose customer number is given by the user. There are about one hundred thousand customers in the customer list in the database. I do not want the method to take the whole list and search it every time it is called, it takes too much effort. However, I don't know how to use a list in where ().
In summary, instead of pulling out all the list I want and searching the values requested by the user in the list, I want to go to the database with the list that comes directly from the user and give me the information of these customers.
I hope I could explain. Thank you for your help.
Try this:
var customers = _context.Customers.Where(c => customerNumber.Contains(c.CustomerId)).Select(i => new SelectedCustomers() { }).ToList()
Try something like (not tested)
public List<SelectedCustomers> GetCustomers(List<int> customerNumbers)
{
var customers=_context.Customers.Where(x =customerNumbers.Contains(x.customerNumber)
.Select(i=> new SelectedCustomers() {}).ToList()
}
This is the equivalent of the SQL IN ()
Related
We're planning to add a Redis-cache to an existing solution.
We have this core entity which is fetched a lot, several times per session. The entity consists of 13 columns where the majority is less than 20 characters. Typically it's retrieved by parent id, but sometimes as a subset that is fetched by a list of ids. To solve this we're thinking of implementing the solution below, but the question is if it's a good idea? Typically the list is around 400 items, but in some cases it could be up to 3000 items.
We would store the instances in the list with this key pattern: EntityName:{ParentId}:{ChildId}, where ParentId and ChildId is ints.
Then to retrieve the list based on ParentId we would call the below method with EntityName:{ParentId}:* as the value of the pattern-argument:
public async Task<List<T>> GetMatches<T>(string pattern)
{
var keys = _multiPlexer.GetServer(_multiPlexer.GetEndPoints(true)[0]).Keys(pattern: pattern).ToArray();
var values = await Db.StringGetAsync(keys: keys);
var result = new List<T>();
foreach (var value in values.Where(x => x.HasValue))
{
result.Add(JsonSerializer.Deserialize<T>(value));
}
return result;
}
And to retrieve a specific list of items we would call the below method with a list of exact keys:
public async Task<List<T>> GetList<T>(string[] keys)
{
var values = await Db.StringGetAsync(keys: keys.Select(x => (RedisKey)x).ToArray());
var result = new List<T>();
foreach (var value in values.Where(x => x.HasValue))
{
result.Add(JsonSerializer.Deserialize<T>(value));
}
return result;
}
The obvious worry here is the amount of objects to deserialize and the performance of System.Text.Json.
A alternative to this would be to store the data twice, both as a list and on it's own, but that would only help in the case where we're fetching by ParentId. We could also only store the data as a list and retrieve it every time only to sometimes use a subset.
Is there a better way to tackle this?
All input is greatly appreciated! Thanks!
Edit
I wrote a small console application to load test the alternatives, fetching 2000 items 100 times took 2020ms with the pattern matching and fetching the list took 1568ms. I think we can live with that difference and go with the pattern matching.
It seems like #Xerillio was right. I did some load testing using hosted services and then it was almost three times slower to fetch the list using the pattern matching, slower then receiving the list directly from SQL. So, to answer my own question if it's a good idea, I would say no it isn't. The majority of the added time was not because of deserialization rather because of fetching the keys using the pattern matching.
Here's the result from fetching 2000 items 100 items in a loop:
Fetch directly from db = 8625ms
Fetch using list of exact keys = 5663ms
Fetch using match = 13098ms
Fetch full list = 5352ms
Say, I have collection People. How should I fetch first 1000 documents that doesn't have a field Phone? As I understand, I should use $exists however I cannot understand how to use it from .NET driver and there is next to no info on that topic on the internet. Any help will be appreciated. Thanks!
Assume your Model Class is Model and colelction name is "Model".
var coll = db.GetCollection<Model>("Model");
var ret = coll.Find(Builders<Model>.Filter.Exists(d => d.Phone, false))
.Limit(1000)
.ToList();
With ToList you will get already loaded list, sometimes it's better to use ToEnumerable and have enumerable to iterate.
I am working on a small expense tracking program. The idea is to have a list that holds Expense objects that can be manipulated and used to perform calculations.
I was able to create the List without issue and populate it with several dummy expenses. My expenses are grouped by category, Expense.expenseType, to allow me to do calculations for analysis so I am trying to make another List that will store category names and relevant calculations values. The list of category names is meant to remove duplicates but so far I've been unsuccessful at populating it.
My approach for creating the List has been to define a Category class that holds only a string parameter for categoryName and a float for categoryTotal although the latter is initialized to 0.00. I then have a For loop that copies the names into the List and a second For loop that removes indexes based on the name once they've been alphabetized. I've tried different variations of this but ultimately I get either an index that is out of bounds or a reduced but still duplicates list of categoryName.
Really hoping to get some advice so I could move forward with the code. I didn't add the actual code since I'm new to C#/VS and figure I may be approaching the problem all wrong.
Edit 1: Based on the feedback I got, the function I am using is below:
public void getCategories(List<Category> passedCategories)
{
passedCategories = passedCategories.GroupBy(Category =>Category.strName)
.Select(gr => new Category
{
strName = gr.Key,
fltTotal = gr.Sum(ex => ex.Value)
});
}
This function is not working, I have a few points I wanted to clarify and I am sure there are others I missed.
Passed categories is a List of Categories that have three parameters - strName, fltTotal and fltPercent. The latter two are currently set to zero when the whole list is populated via a temp Category. The strName is being copied from an Expense List with many more parameters. Since the Category name will repeat in the Expense List, I am trying to remove all duplicates so I can have just the different categories. I took out var since I am passing the List in, should I not have done this? What am I missing?
Thanks again for the help,
Yusif Nurizade
That you need is something like the following. I say something, because I don't see your code and I have to imagine it. For instance, I don't know the name of the property for the amount of expense. I assumed that this is called Value.
// This would be the list of expenses. You have to populate it with data.
var expenses = new List<Expense>();
// Using LINQ you can achieve that you want in a few lines.
// First you group by your data by their categories.
// Then you calculate the total expense for each category.
var statistics = expenses.GroupBy(expense=>expsense.Type)
.Select(gr=> new Category
{
Name = gr.Key,
Total = gr.Sum(ex=>ex.Value)
});
i followed this tutorial for setting up an upload/download to/from sql database.
http://dotnetawesome.blogspot.co.uk/2013/11/how-to-upload-and-download-files-tofrom.html
It works just fine. But i want to modify the populate method so that it will only populate files where the fileid exists within a list that i've stored in session state.
The problem is i've looked around and i can't make any sense of the lambda expressions or work out how to do this.
Basically, i keep a list of the fileIDs in the session, (which is renewed on first page load) so it will only show the files uploaded for that form submission. (it's a claim form)
using (Portal_Entities dc = new Portal_Entities()) {
List<WEBSITE_ATTACHMENTS> allFiles = dc.WEBSITE_ATTACHMENTS.ToList()
rptAttachments.DataSource = allFiles;
rptAttachments.DataBind();
}
I'm guessing i need to put a .Where or .Select on the .ToList here, but i'm not sure.
Like i need a sql type statement where field in ('value','value') where there values come from the list in the session.
Can anyone help?
You could try this one:
// Get the list of ints called fileIDs that you have stored in session.
List<int> ids = (List<int>)Session["fileIDs"];
// Declare an enumeration in which only the WEBSITE_ATTACHMENTS
// with an in contained in ids, will be contained.
List<WEBSITE_ATTACHMENTS> allFiles = dc.WEBSITE_ATTACHMENTS
.Where(x=>ids.Contains(x.fileId));
I have a simple foreach loop that goes through the productID's I have stored in a user's basket and looks up the product's details from the database.
As you can see from my code, what I have at present will return the very last item on screen - as the variable is overwritten within the loop. I'd like to be able to concat this so that I can display the product details for the items only in the basket.
I know I could do something very easy like store only ProductIDs in the repeater I use and onitemdatabound call the database there but I'd like to make just one database call if possible.
Currently I have the following (removed complex joins from example, but if this matters let me know):
IQueryable productsInBasket = null;
foreach (var thisproduct in store.BasketItems)
{
productsInBasket = (from p in db.Products
where p.Active == true && p.ProductID == thisproduct.ProductID
select new
{
p.ProductID,
p.ProductName,
p.BriefDescription,
p.Details,
p.ProductCode,
p.Barcode,
p.Price
});
}
BasketItems.DataSource = productsInBasket;
BasketItems.DataBind();
Thanks for your help!
It sounds like you really want something like:
var productIds = store.BasketItems.Select(x => x.ProductID).ToList();
var query = from p in db.Products
where p.Active && productIds.Contains(p.ProductID)
select new
{
p.ProductID,
p.ProductName,
p.BriefDescription,
p.Details,
p.ProductCode,
p.Barcode,
p.Price
};
In Jon's answer, which works just fine, the IQueryable will however be converted to an IEnumerable, since you call ToList() on it. This will cause the query to be executed and the answer retrieved. For your situation, this may be OK, since you want to retrieve products for a basket, and where the number of products will probably be considerably small.
I am, however, facing a similar situation, where I want to retrieve friends for a member. Friendship depends on which group two members belongs to - if they share at least one group, they are friends. I thus have to retrieve all membership for all groups for a certain member, then retrieve all members from those groups.
The ToList-approach will not be applicable in my case, since that would execute the query each time I want to handle my friends in various ways, e.g. find stuff that we can share. Retrieving all members from the database, instead of just working on the query and execute it at the last possible time, will kill performance.
Still, my first attempt at this situation was to do just this - retrieve all groups I belonged to (IQueryable), init an List result (IEnumerable), then loop over all groups and append all members to the result if they were not already in the list. Finally, since my interface enforced that an IQueryable was to be returned, I returned the list with AsIQueryable.
This was a nasty piece of code, but at least it worked. It looked something like this:
var result = new List<Member>();
foreach (var group in GetGroupsForMember(member))
result.AddRange(group.GroupMembers.Where(x => x.MemberId != member.Id && !result.Contains(x.Member)).Select(groupMember => groupMember.Member));
return result.AsQueryable();
However, this is BAD, since I add ALL shared members to a list, then convert the list to an IQueryable just to satisfy my post condition. I will retrieve all members that are affected from the database, every time I want to do stuff with them.
Imagine a paginated list - I would then just want to pick out a certain range from this list. If this is done with an IQueryable, the query is just completed with a pagination statement. If this is done with an IEnumerable, the query has already been executed and all operations are applied to the in-memory result.
(As you may also notice, I also navigate down the entity's relations (GroupMember => Member), which increases coupling can cause all kinds of nasty situations further on. I wanted to remove this behavior as well).
So, tonight, I took another round and ended up with a much simpler approach, where I select data like this:
var groups = GetGroupsForMember(member);
var groupMembers = GetGroupMembersForGroups(groups);
var memberIds = groupMembers.Select(x => x.MemberId);
var members = memberService.GetMembers(memberIds);
The two Get methods honor the IQueryable and never convert it to a list or any other IEnumerable. The third line just performs a LINQ query ontop of the IEnumerable. The last line just takes the member IDs and retrieves all members from another service, which also works exclusively with IQueryables.
This is probably still horrible in terms of performance, but I can optimize it further later on, if needed. At least, I avoid loading unnecessary data.
Let me know if I am terribly wrong here.