So I have a little issue in sorting some data I have. In a Telerik Grid, I have a column called Requestor that displays the name of a person or Unit (group of people). The problem is, Requestor has two sources it can get it's data from. Here are the two sources.
1.) RequestorId: This is a foreign key to a table called Customer. Here, I store all the data for the user, including their full name. This field can be null btw.
2.) UnitId: This is another foreign key to a table called Units. Here, I store all the data for the Units, particularlly their names. This field can be null btw.
Here is the logic:
//Entity class that contains all the data for my grid
var purchaseOrders = _purchaseOrders.GetPurchaseOrders();
//Key is Id of PurchaseOrders, Value is name of requestor
var dictionary = new Dictionary<int, string>();
foreach (var purchaseOrder in purchaseOrders)
{
if (purchaseOrder.requestorId != null)
dictionary.add(purchaseOrder.Requestor.Fullname);
else
dictionary.add(purchaseOrder.unit.Fullname);
}
dictionary.orderby(x => x.value).ToDictionary(x => x.Key, x.Value);
var tempPurchaseOrders = new List<PurchaseOrder>();
foreach (var item in dictionary)
{
tempPurchaseOrders.Add(purchaseOrders.Where(x => x.Id == item.Key).FirstOrDefault());
}
purchaseOrders = tempPurchaseOrders.AsQueryable();
return purchaseOrders;
This logic returns an ordered list based on what I want to do, however, the problem is the amount of time it takes to process. It takes 1 minute to process. That's horrible obviously. Is there anyway to optimize this? I cut down the source after I return for the grid because there is no logical way to really cut it down beforehand.
Any help would be appreciated. Thanks.
Edit: I found out I no longer am required to use the RequestName field. That limits the data to two areas now. Still a minute to process though.
Did you try something like this:
return _purchaseOrders.GetPurchaseOrders().Select(i => new
{
OrderColumn = i.requestorId != null ? purchaseOrder.Requestor.Fullname : purchaseOrder.unit.Fullname,
// map other columns
})
.OrderBy(i => i.OrderColumn);
A bit like Sławomir Rosiek's solution (but entity framework won't accept that statement):
return _purchaseOrders.GetPurchaseOrders()
.OrderBy(o => o.unit.Fullname).ToList();
(since you don't use RequestName anymore).
Especially when GetPurchaseOrders() is an IQueryable from EF you delegate the sorting to the database engine because the sort expression becomes part of the SQL statement.
So I came up with my own solution. I first tried what both Sławomir Rosiek and Gert Arnold did. Unfortunately, like Gert mentioned, the first answer would not go through. The second one had similar issues.
In the end, I created a class to store the data from both Requestors and Units. It consisted of the following:
internal class RequestorData
{
public int entityId { get; set; }
public string Name { get; set; }
public bool isRequestorId { get; set; }
}
Next, I did the following.
//Entity class that contains all the data for my grid
var purchaseOrders = _purchaseOrders.GetPurchaseOrders();
var tempPurchaseOrders = new List<PurchaseOrder>();
var requestors = new List<RequestorData>();
var customers = purchaseOrders.Select(po => po.Requestor).Distinct().ToList();
var units = purchaseOrders.Select(po => po.Unit).Distinct().ToList();
foreach (var customer in customers)
{
if (customer != null)
requestors.Add(new RequestorData { entityId = customer.Id, Name = customer.FullName, isRequestorId = true });
}
foreach (var unit in units)
{
if (unit != null)
requestors.Add(new RequestorData { entityId = unit.Id, Name = unit.FullName, isRequestorId = false });
}
requestors = requestors.OrderBy(r => r.Name).ToList();
foreach (var requestor in requestors)
{
var id = requestor.entityId;
if (requestor.isRequestorId)
tempPurchaseOrders.AddRange(purchaseOrders.Where(po => po.RequestorId == id).ToList());
else
tempPurchaseOrders.AddRange(purchaseOrders.Where(po => po.UnitId == id).ToList());
}
purchaseOrders = tempPurchaseOrders.AsQueryable();
return purchaseOrders;
I ran this new rendition and have a 5-6 second time of wait. That's not perfect but much better than before. Thanks for all the help.
Related
I don't have a problem currently, but I want to make sure, that the performance is not too shabby for my issue. My search on Microsofts documentation was without any success.
I have a Entity of the name Reservation. I now want to add some statistics to the program, where I can see some metrics about the reservations (reservations per month and favorite spot/seat in particular).
Therefore, my first approach was the following:
public async Task<ICollection<StatisticElement<Seat>>> GetSeatUsage(Company company)
{
var allReservations = await this.reservationService.GetAll(company);
return await this.FetchGroupedSeatData(allReservations, company);
}
public async Task<ICollection<StatisticElement<DateTime>>> GetMonthlyReservations(Company company)
{
var allReservations = await this.reservationService.GetAll(company);
return this.FetchGroupedReservationData(allReservations);
}
private async Task<ICollection<StatisticElement<Seat>>> FetchGroupedSeatData(
IEnumerable<Reservation> reservations,
Company company)
{
var groupedReservations = reservations.GroupBy(r => r.SeatId).ToList();
var companySeats = await this.seatService.GetAll(company);
return (from companySeat in companySeats
let groupedReservation = groupedReservations.FirstOrDefault(s => s.Key == companySeat.Id)
select new StatisticElement<Seat>()
{
Value = companySeat,
StatisticalCount = groupedReservation?.Count() ?? 0,
}).OrderByDescending(s => s.StatisticalCount).ToList();
}
private ICollection<StatisticElement<DateTime>> FetchGroupedReservationData(IEnumerable<Reservation> reservations)
{
var groupedReservations = reservations.GroupBy(r => new { Month = r.Date.Month, Year = r.Date.Year }).ToList();
return groupedReservations.Select(
groupedReservation => new StatisticElement<DateTime>()
{
Value = new DateTime(groupedReservation.Key.Year, groupedReservation.Key.Month, 1),
StatisticalCount = groupedReservation.Count(),
}).
OrderBy(s => s.Value).
ToList();
}
To explain the code a little bit: With GetSeatUsage and GetMonthlyReservations I can get the above mentioned data of a company. Therefore, I fetch ALL reservations at first (with reservationService.GetAll) - this is the point, where I think the performance will be a problem in the future.
Afterwards, I call either FetchGroupedSeatData or FetchGroupedReservationData, which first groups the reservations I previously fetched from the database and then converts them in a, for me, usable format.
As I said, I think the group by after I have read ALL the data from the database MIGHT be a problem, but I cannot find any information regarding performance in the documentation.
My other idea was, that I create a new method in my ReservationService, which then already returns the grouped list. But, again, I can't find the information, that the EF adds the GroupBy to the DB Query or basically does it after all of the data has been read from the database. This method would look something like this:
return await this.Context.Set<Reservation>.Where(r => r.User.CompanyId == company.Id).GroupBy(r => r.SeatId).ToListAsync();
Is this already the solution? Where can I check that? Am I missing something completely obvious?
I am trying to get a list whose type is a model(called ItemDetail). However, I get this error: "Object reference not set to an instance of an object."
public class ItemDetail
{
public decimal sum { get; set; }
public string username { get; set; }
public string units { get; set; }
public string remarks { get; set; }
}
The API code is as follows:
[HttpGet]
[Route("api/items/details/{id}")]
public IEnumerable<ItemDetail> ItemDetails(int id)
{
using (ShopEntities entities = new ShopEntities())
{
var itemDetails = entities.vwItemDetails.ToList();
var userIds = from data in itemDetails
select data.Userid;
var item_Details = new List<ItemDetail> ();
foreach (int userId in userIds)
{
int totals = (int)entities.vwItemDetails
.Where(i => i.UserID == userId & i.item_id == id)
.Select(i => i.quantity)
.DefaultIfEmpty(0)
.Sum();
var itemRecord = entities.vwItemDetails.SingleOrDefault(i => i.item_id == id & i.Userid == userId);
item_Details.Append(new ItemDetail
{
username = itemRecord.username,
units = itemRecord.units,
remarks = itemRecord.Remarks,
sum = totals
});
}
return item_Details;
}
}
Thanks in advance for your help.
EDIT: The error occurs inside the foreach loop where I'm trying to append the new ItemDetail (item_Details.Append(new ItemDetail)
I think I see the problem...
var itemRecord = entities.vwItemDetails.SingleOrDefault(i => i.item_id == id & i.Userid == userId);
Your filtering for the SingleOrDefault() is using a bitwise AND operator instead of boolean one. The value of itemRecord as it's written right now is almost certainly always null. Try changing that line to this:
var itemRecord = entities.vwItemDetails.SingleOrDefault(i => i.item_id == id && i.Userid == userId);
EDIT:
I just realized you do the same thing in this LINQ area:
int totals = (int)entities.vwItemDetails
.Where(i => i.UserID == userId & i.item_id == id)
.Select(i => i.quantity)
.DefaultIfEmpty(0)
.Sum();
Again, totals is probably coming up as 0.
EDIT 2:
There is more wrong here than I original anticipated. I created a semi-complete working sample and you've got problems beyond the use of the bitwise operator.
As orhtej2 pointed out in the comments, you are setting yourself up to return null, but you don't check for it. So that is the immediate cause of your exception. You're probably iterating through a list of user IDs where some of the IDs aren't linked to the item ID you're working with. That will return a null because of SingleOrDefault.
The fix for that is to check if itemRecord is null, and if so, do continue; to move onto the next user ID. Or if you want to stop processing and return an error, do that. Either way the situation should be handled.
Related to that is another consequence of using SingleOrDefault. A friend of mine pointed out that if you end up with more than one result in your where clause there, you will get an exception as well. Unless you can guarantee that no single user ID will have more than one instance of a given item ID in that collection of item details, you need to use a different method. The most straightforward would be to use Where() instead, and handle the IEnumerable<> that it returns. You'll have another loop, but that's showbiz.
This is purely to improve my skill. My solution works fine for the primary task, but it's not "neat". I'm currently working on a .NET MVC with Entity framework project. I know only basic singular LINQ functions which have sufficed over the years. Now I'd like to learn how to fancy.
So I have two models
public class Server
{
[Key]
public int Id { get; set; }
public string InstanceCode { get; set; }
public string ServerName { get; set; }
}
public class Users
{
[Key]
public int Id { get; set; }
public string Name { get; set; }
public int ServerId { get; set; } //foreign key relationship
}
In one of my view models I was asked to provide a dropdown list for selecting a server when creating a new user. The drop down list populated with text and value Id as an IEnumerable
Here's my original property for dropdown list of servers
public IEnumerable<SelectListItem> ServerItems
{
get { Servers.ToList().Select(s => new selectListItem { Value = x.Id.ToString(), Text = $"{s.InstanceCode}#{s.ServerName}" }); }
}
Update on requirements, now I need to display how many users are related to each server selection. Ok no problem. Here's what I wrote off the top of my head.
public IEnumerable<SelectListItem> ServerItems
{
get
{
var items = new List<SelectListItem>();
Servers.ToList().ForEach(x => {
var count = Users.ToList().Where(t => t.ServerId == x.Id).Count();
items.Add(new SelectListItem { Value = x.Id.ToString(), Text = $"{x.InstanceCode}#{x.ServerName} ({count} users on)" });
});
return items;
}
}
This gets my result lets say "localhost#rvrmt1u (8 Users)" but thats it..
What if I want to sort this dropdown list by user count. All I'm doing is another variable in the string.
TLDR ... I'm sure that someone somewhere can teach me a thing or two about converting this to a LINQ Query and making it look nicer. Also bonus points for knowing how I could sort the list to show servers with the most users on it first.
OK, we have this mess:
var items = new List<SelectListItem>();
Servers.ToList().ForEach(x => {
var count = Users.ToList().Where(t => t.ServerId == x.Id).Count();
items.Add(new SelectListItem { Value = x.Id.ToString(), Text = $"{x.InstanceCode}#{x.ServerName} ({count} users on)" });
});
return items;
Make a series of small, careful, obviously-correct refactorings that gradually improve the code.
Start with: Let's abstract those complicated operations to their own methods.
Note that I've replaced the unhelpful x with the helpful server.
int UserCount(Server server) =>
Users.ToList().Where(t => t.ServerId == server.Id).Count();
Why on earth is there a ToList on Users? That looks wrong.
int UserCount(Server server) =>
Users.Where(t => t.ServerId == server.Id).Count();
We notice that there is a built-in method that does these two operations together:
int UserCount(Server server) =>
Users.Count(t => t.ServerId == server.Id);
And similarly for creating an item:
SelectListItem CreateItem(Server server, int count) =>
new SelectListItem
{
Value = server.Id.ToString(),
Text = $"{server.InstanceCode}#{server.ServerName} ({count} users on)"
};
And now our property body is:
var items = new List<SelectListItem>();
Servers.ToList().ForEach(server =>
{
var count = UserCount(server);
items.Add(CreateItem(server, count);
});
return items;
Already much nicer.
Never use ForEach as a method if you're just going to pass a lambda body! There's already a built-in mechanism in the language that does it better! There is no reason to write items.Foreach(item => {...}); when you could simply write foreach(var item in items) { ... }. It's simpler and easier to understand and debug, and the compiler can optimize it better.
var items = new List<SelectListItem>();
foreach (var server in Servers.ToList())
{
var count = UserCount(server);
items.Add(CreateItem(server, count);
}
return items;
Much nicer.
Why is there a ToList on Servers? Completely unnecessary!
var items = new List<SelectListItem>();
foreach(var server in Servers)
{
var count = UserCount(server);
items.Add(CreateItem(server, count);
}
return items;
Getting better. We can eliminate the unnecessary variable.
var items = new List<SelectListItem>();
foreach(var server in Servers)
items.Add(CreateItem(server, UserCount(server));
return items;
Hmm. This gives us an insight that CreateItem could be doing the count itself. Let's rewrite it.
SelectListItem CreateItem(Server server) =>
new SelectListItem
{
Value = server.Id.ToString(),
Text = $"{server.InstanceCode}#{server.ServerName} ({UserCount(server)} users on)"
};
Now our prop body is
var items = new List<SelectListItem>();
foreach(var server in Servers)
items.Add(CreateItem(server);
return items;
And this should look familiar. We have re-invented Select and ToList:
var items = Servers.Select(server => CreateItem(server)).ToList();
Now we notice that the lambda can be replaced with the method group:
var items = Servers.Select(CreateItem).ToList();
And we have reduced that whole mess to a single line that clearly and unambiguously looks like what it does. What does it do? It creates an item for every server and puts them in a list. The code should read like what it does, not how it does it.
Study the techniques I used here carefully.
Extract complex code to helper methods
Replace ForEach with real loops
Eliminate unnecessary ToLists
Revisit earlier decisions when you realize there's an improvement to be made
Recognize when you are re-implementing simple helper methods
Don't stop with one improvement! Each improvement makes it possible to do another.
What if I want to sort this dropdown list by user count?
Then sort it by user count! We abstracted that away into a helper method, so we can use it:
var items = Servers
.OrderBy(UserCount)
.Select(CreateItem)
.ToList();
We now notice that we're calling UserCount twice. Do we care? Maybe. It could be a perf problem to call it twice, or, horrors, it might not be idempotent! If either are a problem then we need to undo a decision we made before. It's easier to deal with this situation in comprehension mode rather than fluent mode, so let's rewrite as a comprehension:
var query = from server in Servers
orderby UserCount(server)
select CreateItem(server);
var items = query.ToList();
Now we go back to our earlier:
SelectListItem CreateItem(Server server, int count) => ...
and now we can say
var query = from server in Servers
let count = UserCount(server)
orderby count
select CreateItem(server, count);
var items = query.ToList();
and we are only calling UserCount once per server.
Why go back to comprehension mode? Because to do this in fluent mode makes a mess:
var query = Servers
.Select(server => new { server, count = UserCount(server) })
.OrderBy(pair => pair.count)
.Select(pair => CreateItem(pair.server, pair.count))
.ToList();
And it looks a little ugly. (In C# 7 you could use a tuple instead of an anonymous type, but the idea is the same.)
The trick with LINQ is just to type return and go from there. Don't create a list and add items to it; there is usually a way to select it all in one go.
public IEnumerable<SelectListItem> ServerItems
{
get
{
return Servers.Select
(
server =>
new
{
Server = server,
UserCount = Users.Count( u => u.ServerId = server.Id )
}
)
.Select
(
item =>
new SelectListItem
{
Value = item.Server.Id.ToString(),
Text = string.Format
(
#"{0}{1} ({2} users on)" ,
item.Server.InstanceCode,
item.Server.ServerName,
item.UserCount
)
}
);
}
}
In this example there are actually two Select statements-- one to extract the data, and one to do the formatting. In an ideal situation the logic for those two tasks would be separated into different layers, but this is an OK compromise.
i'm sorry if this question has already been asked, but i'm in trouble with my method of updating collection in Entity Framework.
Let me explain the situation :
- I have for example one model CUSTOMER with some properties and a collection of ORDERS (for example).
- Let's imagine we have an admin page on wich we can edit all the ORDERS for a customer, and when we submit the form, it will send us back the object CUSTOMERS with updated ORDERS (some added, some updated and some deleted).
For the moment i use something like this in order to compare old collection and new collection and determine which object i need to delete/update/add
var toRemove = new List<ORDERS>();
var toAdd = new List<ORDERS>();
foreach (
var order in
oldList.Where(
order =>
newList.FirstOrDefault(t => t.link_id == order.link_id) == null))
{
toRemove.Add(order);
}
foreach (
var order in
newList.Where(
order =>
oldList.FirstOrDefault(t => t.link_id == order.link_id) == null))
{
toAdd.Add(order);
}
foreach (var ORDERSe in toRemove)
{
bdd.ORDERS.Remove(ORDERSe);
}
foreach (var ORDERSe in toAdd)
{
ORDERSe.pjt_id = project_id;
bdd.ORDERS.Add(ORDERSe);
}
foreach (
var order in
newList.Where(
order =>
oldList.FirstOrDefault(t => t.link_id == order.link_id) != null))
{
var child = oldList.FirstOrDefault(t => t.link_id == order.link_id);
bdd.Entry(child).CurrentValues.SetValues(order);
}
But i'm unconfortable with this, because in my mind, entity framework should be able to do the work for me !
I was hoping something like :
customer.orders = newOrders;
Did i missed anything about entity framework or ?
Because when i do this, it just duplicate my orders.
Thanks in advance for your answer.
You can certainly make it cleaner using .Except() and .Intersect(), but the concept doesn't really change, AFAIK you still have to individually remove, update & add the entries in loops...
var oldList = new List<ORDERS>();
var newList= new List<ORDERS>();
var IdsToRemove = oldList.Select(t => t.link_id).Except(newList.Select(t => t.link_id));
var IdsToAdd = newList.Select(t => t.link_id).Except(oldList.Select(t => t.link_id));
var IdsToUpdate = newList.Select(t => t.link_id).Intersect(oldList.Select(t => t.link_id));
//remove
bdd.orders.where(x => IdsToRemove.Contains(x.link_id)).ForEach(x => bdd.Remove(x));
//add
foreach(var order in newList.Where(x -> IdsToAdd.Contains(x.link_id))
{
bdd.Orders.Attach(order);
bdd.Entries(order).EntityState = EntityState.Added;
}
//update
foreach(var order in newList.Where(x -> IdsToUpdate .Contains(x.link_id))
{
bdd.Orders.Attach(order);
bdd.Entries(order).EntityState = EntityState.Modified;
}
bdd.SaveChanges();
But i'm unconfortable with this, because in my mind, entity framework
should be able to do the work for me !
In fact, EF does the Work for you. Using the data context SaveChanges method EF should be able to save all your changes at once:
DbContext.SaveChanges()
For your convinience you can still override this method. Internally you should use something like this:
public override int SaveChanges()
{
var changeSet = ChangeTracker.Entries<IAuditable>();
if (changeSet != null)
{
foreach (var entry in changeSet.Where(c => c.State != EntityState.Unchanged))
{
entry.Entity.ModifiedDate = DateProvider.GetCurrentDate();
entry.Entity.ModifiedBy = UserName;
}
}
return base.SaveChanges();
}
I have a method that updates a ReportRecipient object in EF. The primitives work fine; the headache comes in when trying to manage a M2M relationship with the RecipientGroups objects.
Please take a look at this code:
public IReportRecipient ModifyRecipientWithGroupAssignments(IEnumerable<Guid> groupIds, IReportRecipient recipient)
{
var entity = Context.ReportRecipients
.Include("RecipientGroups")
.FirstOrDefault(e => e.ReportRecipientId == recipient.ReportRecipientId)
.FromIReportRecipient(recipient);
var toRemove = entity.RecipientGroups
.Where(e => !groupIds.Contains(e.GroupId))
.ToList();
//remove group assignments that no longer apply
foreach (var group in toRemove)
{
if (group != null)
{
entity.RecipientGroups.Attach(group);
entity.RecipientGroups.Remove(group);
}
}
var toAdd = entity.RecipientGroups
.Where(e => groupIds.Contains(e.GroupId))
.ToList();
//add new groups that weren't there before
foreach (var group in toAdd)
{
if (group != null)
{
entity.RecipientGroups.Attach(group);
}
}
return entity;
}
... my problem is on the var ToAdd... line. Even if I have a collection of Guids in groupIds that match Guids representing RecipientGroup objects in the database, toAdd always evaluates to an empty collection. I would think the Contains() function would work for this scenario; can someone please explain if I am doing something wrong?
You should load the RecipientGroups you want to add from the database (Context.RecipientGroups I guess), not from the collection you want to add them to (entity.RecipientGroups in the code sample).