LINQ to Entities Select taking a long time - c#

I have a People table with around 1000 rows and a Dramas table with around 100 rows in SQL Azure. The two are linked with a foreign key Drama.PersonId so that each person can have 0 or more dramas.
The code below behaves as expected, returning around 50 people and their associated recent dramas. However, it takes more than 5 seconds to run (measured with a Stopwatch). There must be something inefficient going on?
var people = ctx.People
.Where(p => p.Dramas.Any(d => d.DateHappened >= startDate))
.Select(p => new
{
p.FirstName,
p.LastName,
Dramas = p.Dramas.Where(d => d.DateHappened >= startDate).Select(d => new { d.Id, d.DramaType })
}).AsEnumerable();

I've made this much faster by first fetching all recent dramas and then sending a separate query to get the people. It uses a PredicateBuilder.
var dramasByPerson = ctx.Dramas.Where(d => d.DateHappened >= startDate)
.Select(d => new { d.PersonId, d.Id, d.DramaType })
.ToLookup(d => d.PersonId);
var predicate = dramasByPerson.Select(o => o.Key)
.Aggregate(
PredicateBuilder.False<Person>(),
(current, personId) => current.Or(o => o.PersonId == personId)
);
var dictPeople = ctx.People.Where(predicate)
.Select(o => new { o.PersonId, o.LastName, o.FirstName })
.ToDictionary(o => o.PersonId);
var people = dramasByPerson.Select(o => new {
LastName = people[o.Key].LastName,
FirstName = people[o.Key].FirstName,
Dramas = o.Select(d => new { d.Id, d.DramaType })
});

Related

Joining and selecting only the first item based on clause

I have a Customers and an Orders database.
I need to make some statistics for the first order of all new customers and count the number of first orders from new clients by month.`
var date = new DateTime(now.Year - 1, now.Month, 1);
db.Orders
.Where(o => o.Customer.IsNew && o.OrderDate > date)
.GroupBy(o => new { o.OrderDate.Year, o.OrderDate.Month })
.Select(g => new NewCustomerStatsModel {
Month = g.Key.Month,
Year = g.Key.Year,
Count = g.Count()
})
.OrderBy(cs => cs.Year)
.ThenBy(cs => cs.Month)
.ToList();
This query provide me the number of orders for all new client but I need to get only the sum of the first order for each new Customer if the first order date is greater than the provided date.
Is it possible to do it with a query (and how) or am I forced to use AsEnumerable and do it in memory?
I need to make some statistics for the first order of all new customers
var clientFirstOrders = db.Customers.Where(c => c.IsNew)
.Select(c => new{
Customer = c,
FirstOrder = c.Orders.OrderBy(c => c.OrderDate).FirstOrDefault()
})
// might have to do (int?)FirstOrder.Id != null or something like that.
.Where(e => e.FirstOrder != null);
and count the number of first orders from new clients by month.
var clientCountByFirstOrderMonth = clientFirstOrders
.GroupBy(e => new { e.FirstOrder.OrderDate.Year, e.FirstOrder.OrderDate.Month })
.Select(g => new{g.Key.Year, g.Key.Month, Count = g.Count()});
I could find the solution.
With some appropriate index, the performances are pretty good.
It's probably not a perfect solution, but I couldn't update the entities because it's not my Library.
var date = new DateTime(now.Year - 1, now.Month, 1);
var result = db.Orders
.Where(o => o.Customer.IsNew && o.State != OrderState.Cancelled) // get all orders where the Customer is a new one.
.GroupBy(o => o.Customer.Id) // group by customer
.Select(g => g.OrderBy(o => o.OrderDate).FirstOrDefault()) // get the first order for every customer
.Where(o => o.OrderDate > date) // restrict to the given date
.GroupBy(o => new { o.OrderDate.Year, o.OrderDate.Month) }) // then group by month
.Select(g => new NewCustomerStatsModel {
Month = g.Key.Month,
Year = g.Key.Year,
Count = g.Count()
})
.OrderBy(g => g.Year)
.ThenBy(g => g.Month)
.ToList();

Simplifying a Linq query by using Include instead of Join

I've been asked to simplify the following Linq query:
var orders = db.Orders
.Join(db.Shipments,
o => o.OrderID,
s => s.OrderID,
(o, s) => new { Order = o, Shipment = s })
.Join(db.LineItems,
s => s.Shipment.ShipmentID,
l => l.ShipmentID,
(s, l) => new { Order = s.Order, Shipment = s.Shipment, LineItem = l })
.Join(db.StatusTypes,
s => s.Shipment.StatusTypeID,
st => st.StatusTypeID,
(s, st) => new { Order = s.Order, Shipment = s.Shipment, LineItem = s.LineItem, Description = st.ExternalDescription })
.Where(x => x.Order.AccountID == accountId)
.GroupBy(x => x.Order.OrderNumber)
.ToList()
.Select(
x => new OrderStatusViewModel
{
Date = x.Max(y => y.Order.Created),
OrderNumber = x.Key,
Cost = x.Sum(y => y.LineItem.UnitPrice).ToString(),
Status = x.Max(y => y.Description)
}
);
By replacing the Joins with Includes. I've searched around, and I've discovered that Joins and Includes are somewhat equivalent. But I can't figure out how to convert this query to one that uses includes instead of joins. Is it actually less code and simpler to use includes instead of joins here?
I strongly suggest you to use navigation properties instead of manual builded joins. It will be more efficent and controlable. Read this article.
If you would convert your query to navigation property form by using Include, it would seem like that;
var orders = db.Orders
.Include(x => x.Shipments)
.Include(x => x.Shipments.Select(y => y.LineItems))
.Include(x => x.Shipments.Select(y => y.StatusType))
.Where(x => x.Order.AccountID == accountId)
.GroupBy(x => x.Order.OrderNumber)
.ToList()
.Select(
x => new OrderStatusViewModel
{
Date = x.Max(y => y.Created),
OrderNumber = x.Key,
Cost = x.LineItems.Sum(k => k.UnitPrice),
Status = x.Max(y => y.Description)
}
);
But, as I said, you should define the navigation properties for entities first.
var orders = db.Orders
.Include("Shipments")
.Include("Shipments.LineItems")
.Include("Shipments.StatusTypes")
.Where(x => x.Order.AccountID == accountId)
.GroupBy(x => x.Order.OrderNumber)
.ToList()
.Select(
x => new OrderStatusViewModel
{
Date = x.Max(y => y.Order.Created),
OrderNumber = x.Key,
Cost = x.Sum(y => y.LineItem.UnitPrice).ToString(),
Status = x.Max(y => y.Description)
}
);
i not tested above code ,just try
ObjectQuery.Include Method (String)

Comparing list with list inside another list with linq [duplicate]

I am getting this error for the query below
Unable to create a constant value of type API.Models.PersonProtocol. Only primitive types or enumeration types are supported in this context
ppCombined below is an IEnumerable object of PersonProtocolType, which is constructed by concat of 2 PersonProtocol lists.
Why is this failing? Can't we use LINQ JOIN clause inside of SELECT of a JOIN?
var persons = db.Favorites
.Where(x => x.userId == userId)
.Join(db.Person, x => x.personId, y => y.personId, (x, y) =>
new PersonDTO
{
personId = y.personId,
addressId = y.addressId,
favoriteId = x.favoriteId,
personProtocol = (ICollection<PersonProtocol>) ppCombined
.Where(a => a.personId == x.personId)
.Select( b => new PersonProtocol()
{
personProtocolId = b.personProtocolId,
activateDt = b.activateDt,
personId = b.personId
})
});
This cannot work because ppCombined is a collection of objects in memory and you cannot join a set of data in the database with another set of data that is in memory. You can try instead to extract the filtered items personProtocol of the ppCombined collection in memory after you have retrieved the other properties from the database:
var persons = db.Favorites
.Where(f => f.userId == userId)
.Join(db.Person, f => f.personId, p => p.personId, (f, p) =>
new // anonymous object
{
personId = p.personId,
addressId = p.addressId,
favoriteId = f.favoriteId,
})
.AsEnumerable() // database query ends here, the rest is a query in memory
.Select(x =>
new PersonDTO
{
personId = x.personId,
addressId = x.addressId,
favoriteId = x.favoriteId,
personProtocol = ppCombined
.Where(p => p.personId == x.personId)
.Select(p => new PersonProtocol
{
personProtocolId = p.personProtocolId,
activateDt = p.activateDt,
personId = p.personId
})
.ToList()
});
In my case, I was able to resolve the issue by doing the following:
I changed my code from this:
var r2 = db.Instances.Where(x => x.Player1 == inputViewModel.InstanceList.FirstOrDefault().Player2 && x.Player2 == inputViewModel.InstanceList.FirstOrDefault().Player1).ToList();
To this:
var p1 = inputViewModel.InstanceList.FirstOrDefault().Player1;
var p2 = inputViewModel.InstanceList.FirstOrDefault().Player2;
var r1 = db.Instances.Where(x => x.Player1 == p1 && x.Player2 == p2).ToList();
Don't know if anyone searches for this.
I had the same problem. A select on the query and then doing the where (or join) and using the select variable solved the problem for me.
(problem was in the collection "Reintegraties" for me)
query.Select(zv => new
{
zv,
rId = zv.this.Reintegraties.FirstOrDefault().Id
})
.Where(x => !db.Taken.Any(t => t.HoortBijEntiteitId == x.rId
&& t.HoortBijEntiteitType == EntiteitType.Reintegratie
&& t.Type == TaakType))
.Select(x => x.zv);
hope this helps anyone.
I had this issue and what I did and solved the problem was that I used AsEnumerable() just before my Join clause.
here is my query:
List<AccountViewModel> selectedAccounts;
using (ctx = SmallContext.GetInstance()) {
var data = ctx.Transactions.
Include(x => x.Source).
Include(x => x.Relation).
AsEnumerable().
Join(selectedAccounts, x => x.Source.Id, y => y.Id, (x, y) => x).
GroupBy(x => new { Id = x.Relation.Id, Name = x.Relation.Name }).
ToList();
}
I was wondering why this issue happens, and now I think It is because after you make a query via LINQ, the result will be in memory and not loaded into objects, I don't know what that state is but they are in in some transitional state I think. Then when you use AsEnumerable() or ToList(), etc, you are placing them into physical memory objects and the issue is resolving.
It's worth adding, since the OP's code sample doesn't provide enough context to prove otherwise, but I received this error as well on the following code:
public RetailSale GetByRefersToRetailSaleId(Int32 refersToRetailSaleId)
{
return GetQueryable()
.FirstOrDefault(x => x.RefersToRetailSaleId.Equals(refersToRetailSaleId));
}
Apparently, I cannot use Int32.Equals in this context to compare an Int32 with a primitive int; I had to (safely) change to this:
public RetailSale GetByRefersToRetailSaleId(Int32 refersToRetailSaleId)
{
return GetQueryable()
.FirstOrDefault(x => x.RefersToRetailSaleId == refersToRetailSaleId);
}
Just add AsEnumerable() andToList() , so it looks like this
db.Favorites
.Where(x => x.userId == userId)
.Join(db.Person, x => x.personId, y => y.personId, (x, y).ToList().AsEnumerable()
ToList().AsEnumerable()

Performing multiple Linq queries against the same Linq result

I have created a dashboard that all data displayed on it shares 4 common elements (startDate,endDate,CompanyID,StoreID) that are used as Where clauses in a Linq statement. The result of that statement is then queried in a variety of ways to group and sort the data and used in charts, lists etc. Here is a short snippit to show the duplication that is currently going on:
var dashboardEntity = new BlueStreakSalesDWEntities();
//Get Total Sales
ViewBag.companySalesTotal = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate)
.Where(d => d.DateKey <= endDate)
.Where(c => c.CompanyID == companyID)
.Sum(a => a.Amount);
//get list of all items sold
var companyStoreTotalItem = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate)
.Where(d => d.DateKey <= endDate)
.Where(c => c.CompanyID == companyID).GroupBy(m => new { m.Description })
.Select(g => new DescriptionAmountModel { Amount = g.Sum(a => a.Amount).Value, Description = g.Key.Description })
.OrderByDescending(x => x.Amount);
I have like 15 of these calls on the dashboard and it can get very slow at times from what I imagine are multiple calls when in reality the database only needs to be queried once then that result needs to be queried for different results.
How can I do this?
Any help would be greatly appreciated
In your current solution each query executes separatly, on the same data. You can first execute the shared parts of the queries and bring the results from database. In your examples it is these where conditions
//Executes in database
var entities = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate)
.Where(d => d.DateKey <= endDate)
.Where(c => c.CompanyID == companyID)
.ToList();
Now that this data is filtered to only what you want you can in memory do the rest of the aggregations:
//Happens in the List<T> in memory
ViewBag.companySalesTotal = entities.Sum(a => a.Amount);
var companyStoreTotalItem = entities.GroupBy(m => new { m.Description })
.Select(g => new DescriptionAmountModel { Amount = g.Sum(a => a.Amount).Value, Description = g.Key.Description })
.OrderByDescending(x => x.Amount);
This way you can make efficient. This make the query execute single time in database and rest of the part happen on the pullout in memory data
var result = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate && d => d.DateKey <= endDate && d.CompanyID == companyID).ToList();
ViewBag.companySalesTotal = result.Sum(a => a.Amount);
//then get list of all items sold from in memory data
var companyStoreTotalItem = result.GroupBy(m => new { m.Description }).Select(g => new DescriptionAmountModel { Amount = g.Sum(a => a.Amount).Value, Description = g.Key.Description }).OrderByDescending(x => x.Amount);

How can I transform this SQL query to LINQ?

How can I transform this SQL query to LINQ?
SELECT eg.Name Name, sum(bi.PlannedAmount) Amount
FROM BudgetItem bi, Expense e, ExpenseGroup eg
WHERE Discriminator = 'ExpenseItem' AND
bi.ExpenseId = e.Id AND
e.ExpenseGroupId = eg.id AND
bi.MonthlyBudgetId = 1
GROUP BY eg.Name
So far I've come up with this line:
var result = context
.ExpenseGroups
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
But I still cannot figure out what expression to use to add 'bi.MonthlyBudgetItem = 1'.
Does anybody have an Idea?
Edit #1:
I forgot to mention the relationships between the entities. Every ExpenseGroup has many Expenses, and every Expense has many BudgetItems.
So, ExpenseGroup => Expenses => BudgetItems
Edit #2:
I'm using Entity Framework and every ExpenseGroup has a Collection of Expense objects (every Expense has a ExpenseGroup object), as well as every Expense has a Collection of BudgetItem objects (every BudgetItem object has a Expense object).
I suppose something like this should do it:
var result = context
.ExpenseGroups
.Where(x => x.Discriminator == 'ExpenseItem' &&
x.bi.ExpenseId == e.Id &&
x.e.ExpenseGroupId == eg.id &&
x.bi.MonthlyBudgetId == 1)
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
Something similar to this...
var result = (from g in context.ExpenseGroups
where g.Expense.BudgetItem.MonthlyBudgetId == 1
select g)
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
or
var result = context.ExpenseGroups
.Where(g => g.Expense.BudgetItem.MonthlyBudgetId == 1)
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
You are actually doing an inner join in your SQL query, so do similarly in your linq query as well. This should work:-
var result = from bi in context.BudgetItem
join e in context.Expense
on bi.ExpenseId equals e.Id
where bi.MonthlyBudgetId == 1
join eg in ExpenseGroup
on e.ExpenseGroupId equals eg.id
group new { bi, eg } by eg.Name into g
select new
{
Name = g.Key,
Amount = g.Sum(x => x.bi.PlannedAmount)
};

Categories

Resources