I have created a dashboard that all data displayed on it shares 4 common elements (startDate,endDate,CompanyID,StoreID) that are used as Where clauses in a Linq statement. The result of that statement is then queried in a variety of ways to group and sort the data and used in charts, lists etc. Here is a short snippit to show the duplication that is currently going on:
var dashboardEntity = new BlueStreakSalesDWEntities();
//Get Total Sales
ViewBag.companySalesTotal = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate)
.Where(d => d.DateKey <= endDate)
.Where(c => c.CompanyID == companyID)
.Sum(a => a.Amount);
//get list of all items sold
var companyStoreTotalItem = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate)
.Where(d => d.DateKey <= endDate)
.Where(c => c.CompanyID == companyID).GroupBy(m => new { m.Description })
.Select(g => new DescriptionAmountModel { Amount = g.Sum(a => a.Amount).Value, Description = g.Key.Description })
.OrderByDescending(x => x.Amount);
I have like 15 of these calls on the dashboard and it can get very slow at times from what I imagine are multiple calls when in reality the database only needs to be queried once then that result needs to be queried for different results.
How can I do this?
Any help would be greatly appreciated
In your current solution each query executes separatly, on the same data. You can first execute the shared parts of the queries and bring the results from database. In your examples it is these where conditions
//Executes in database
var entities = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate)
.Where(d => d.DateKey <= endDate)
.Where(c => c.CompanyID == companyID)
.ToList();
Now that this data is filtered to only what you want you can in memory do the rest of the aggregations:
//Happens in the List<T> in memory
ViewBag.companySalesTotal = entities.Sum(a => a.Amount);
var companyStoreTotalItem = entities.GroupBy(m => new { m.Description })
.Select(g => new DescriptionAmountModel { Amount = g.Sum(a => a.Amount).Value, Description = g.Key.Description })
.OrderByDescending(x => x.Amount);
This way you can make efficient. This make the query execute single time in database and rest of the part happen on the pullout in memory data
var result = dashboardEntity.FactSales.Where(d => d.DateKey >= startDate && d => d.DateKey <= endDate && d.CompanyID == companyID).ToList();
ViewBag.companySalesTotal = result.Sum(a => a.Amount);
//then get list of all items sold from in memory data
var companyStoreTotalItem = result.GroupBy(m => new { m.Description }).Select(g => new DescriptionAmountModel { Amount = g.Sum(a => a.Amount).Value, Description = g.Key.Description }).OrderByDescending(x => x.Amount);
Related
I have a Customers and an Orders database.
I need to make some statistics for the first order of all new customers and count the number of first orders from new clients by month.`
var date = new DateTime(now.Year - 1, now.Month, 1);
db.Orders
.Where(o => o.Customer.IsNew && o.OrderDate > date)
.GroupBy(o => new { o.OrderDate.Year, o.OrderDate.Month })
.Select(g => new NewCustomerStatsModel {
Month = g.Key.Month,
Year = g.Key.Year,
Count = g.Count()
})
.OrderBy(cs => cs.Year)
.ThenBy(cs => cs.Month)
.ToList();
This query provide me the number of orders for all new client but I need to get only the sum of the first order for each new Customer if the first order date is greater than the provided date.
Is it possible to do it with a query (and how) or am I forced to use AsEnumerable and do it in memory?
I need to make some statistics for the first order of all new customers
var clientFirstOrders = db.Customers.Where(c => c.IsNew)
.Select(c => new{
Customer = c,
FirstOrder = c.Orders.OrderBy(c => c.OrderDate).FirstOrDefault()
})
// might have to do (int?)FirstOrder.Id != null or something like that.
.Where(e => e.FirstOrder != null);
and count the number of first orders from new clients by month.
var clientCountByFirstOrderMonth = clientFirstOrders
.GroupBy(e => new { e.FirstOrder.OrderDate.Year, e.FirstOrder.OrderDate.Month })
.Select(g => new{g.Key.Year, g.Key.Month, Count = g.Count()});
I could find the solution.
With some appropriate index, the performances are pretty good.
It's probably not a perfect solution, but I couldn't update the entities because it's not my Library.
var date = new DateTime(now.Year - 1, now.Month, 1);
var result = db.Orders
.Where(o => o.Customer.IsNew && o.State != OrderState.Cancelled) // get all orders where the Customer is a new one.
.GroupBy(o => o.Customer.Id) // group by customer
.Select(g => g.OrderBy(o => o.OrderDate).FirstOrDefault()) // get the first order for every customer
.Where(o => o.OrderDate > date) // restrict to the given date
.GroupBy(o => new { o.OrderDate.Year, o.OrderDate.Month) }) // then group by month
.Select(g => new NewCustomerStatsModel {
Month = g.Key.Month,
Year = g.Key.Year,
Count = g.Count()
})
.OrderBy(g => g.Year)
.ThenBy(g => g.Month)
.ToList();
I want to translate this into lambda syntax and can't seem to get it to work:
Grouping by two columns, select max on a different column, return list of complete complex object.
I am writing more text here to get past the validation on this form. How much text is needed until I am allowed to post this?
_clientpolicies = (from policy in
_reply.CommercialInsuredGroupWithPolicyTerm.InsuredWithPolicyTerm.SelectMany(x => x.PolicyTerm)
.Where(x => !(string.IsNullOrWhiteSpace(x.PolicyNumber) && string.IsNullOrWhiteSpace(x.ControlNumber)))
.Where(x => x.Insured.DNBAccountNumber == _client.LookupID)
group policy by
new
{
PolicyReference = GetPolicyReference(policy),
PolicyType = policy.ProductInformation.PolicyTypeCode
}
into g
let maxPolicyInception = g.Max(p => p.InceptionDate)
from policyGroup in g
where policyGroup.InceptionDate == maxPolicyInception
select policyGroup).ToList();
I dont think there's a way of doing it in one line. So there's my try :
policyGroups=
_reply.CommercialInsuredGroupWithPolicyTerm.InsuredWithPolicyTerm
.SelectMany(x => x.PolicyTerm)
.Where(x => !(string.IsNullOrWhiteSpace(x.PolicyNumber) && string.IsNullOrWhiteSpace(x.ControlNumber)))
.Where(x => x.Insured.DNBAccountNumber == _client.LookupID)
.GroupBy(x => GetPolicyReference(x))
.ThenBy(x => x.ProductInformation.PolicyTypeCode)
.ToList();
var maxPolicyInception = policyGroups.Max(p => p.InceptionDate);
_clientpolicies = policyGroups
.Where(g => g.InceptionDate == maxPolicyInception)
.ToList();
_clientpolicies =
_reply.CommercialInsuredGroupWithPolicyTerm.InsuredWithPolicyTerm.SelectMany(x => x.PolicyTerm)
.Where(x => !(string.IsNullOrWhiteSpace(x.PolicyNumber) && string.IsNullOrWhiteSpace(x.ControlNumber)))
.Where(x => x.Insured.DNBAccountNumber == _client.LookupID)
.GroupBy(x =>
new
{
PolicyReference = GetPolicyReference(x),
PolicyType = x.ProductInformation.PolicyTypeCode
},
(key, g) => g.OrderByDescending(gx => gx.InceptionDate).First()
I have a fairly complicated query that would read from a table, then do group on CONTACT_ID, then select only those group with count of 1.
This query is fairly complicated and I have no idea how to optimize it in LINQ.
var linkTable = this.DB.Links
.Where(l=>l.INSTANCE_ID==123456 && l.CONTACT_ID.HasValue && l.ORGANISATION_ID.HasValue)
.Select(l => new { l.DEFAULT_LINKED_ORGANISATION, l.LINK_ID, l.CONTACT_ID });
var defaultOrganizationLinkQuery = linkTable
.Where(l => l.DEFAULT_LINKED_ORGANISATION)
.Select(l => l.LINK_ID);
var singleOrganizationLinkQuery = linkTable
.GroupBy(l => l.CONTACT_ID)
.Select(group => new
{
CONTACT_ID = group.Key,
contact_link_count = group.Count(),
LINK_ID = group.First().LINK_ID
})
.Where(l => l.contact_link_count == 1)
.Select(l => l.LINK_ID);
var merged = singleOrganizationLinkQuery.Union(defaultOrganizationLinkQuery);
I made shorter version, but I do not expect it to be faster. If it works and is not slower I would be satisfied:
var merged = this.DB.Links
.Where(l=>l.INSTANCE_ID==123456 && l.CONTACT_ID.HasValue && l.ORGANISATION_ID.HasValue)
.GroupBy(l => l.CONTACT_ID)
.SelectMany(s => s.Where(x => s.Count() == 1 || x.DEFAULT_LINKED_ORGANISATION)
.Select(link => link.LINK_ID));
How can I transform this SQL query to LINQ?
SELECT eg.Name Name, sum(bi.PlannedAmount) Amount
FROM BudgetItem bi, Expense e, ExpenseGroup eg
WHERE Discriminator = 'ExpenseItem' AND
bi.ExpenseId = e.Id AND
e.ExpenseGroupId = eg.id AND
bi.MonthlyBudgetId = 1
GROUP BY eg.Name
So far I've come up with this line:
var result = context
.ExpenseGroups
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
But I still cannot figure out what expression to use to add 'bi.MonthlyBudgetItem = 1'.
Does anybody have an Idea?
Edit #1:
I forgot to mention the relationships between the entities. Every ExpenseGroup has many Expenses, and every Expense has many BudgetItems.
So, ExpenseGroup => Expenses => BudgetItems
Edit #2:
I'm using Entity Framework and every ExpenseGroup has a Collection of Expense objects (every Expense has a ExpenseGroup object), as well as every Expense has a Collection of BudgetItem objects (every BudgetItem object has a Expense object).
I suppose something like this should do it:
var result = context
.ExpenseGroups
.Where(x => x.Discriminator == 'ExpenseItem' &&
x.bi.ExpenseId == e.Id &&
x.e.ExpenseGroupId == eg.id &&
x.bi.MonthlyBudgetId == 1)
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
Something similar to this...
var result = (from g in context.ExpenseGroups
where g.Expense.BudgetItem.MonthlyBudgetId == 1
select g)
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
or
var result = context.ExpenseGroups
.Where(g => g.Expense.BudgetItem.MonthlyBudgetId == 1)
.GroupBy(eg => eg.Id, (s) => new { Name = s.Name, Amount = s.Expenses.SelectMany(e => e.Items).Sum(i => i.PlannedAmount) })
.ToList();
You are actually doing an inner join in your SQL query, so do similarly in your linq query as well. This should work:-
var result = from bi in context.BudgetItem
join e in context.Expense
on bi.ExpenseId equals e.Id
where bi.MonthlyBudgetId == 1
join eg in ExpenseGroup
on e.ExpenseGroupId equals eg.id
group new { bi, eg } by eg.Name into g
select new
{
Name = g.Key,
Amount = g.Sum(x => x.bi.PlannedAmount)
};
I have a People table with around 1000 rows and a Dramas table with around 100 rows in SQL Azure. The two are linked with a foreign key Drama.PersonId so that each person can have 0 or more dramas.
The code below behaves as expected, returning around 50 people and their associated recent dramas. However, it takes more than 5 seconds to run (measured with a Stopwatch). There must be something inefficient going on?
var people = ctx.People
.Where(p => p.Dramas.Any(d => d.DateHappened >= startDate))
.Select(p => new
{
p.FirstName,
p.LastName,
Dramas = p.Dramas.Where(d => d.DateHappened >= startDate).Select(d => new { d.Id, d.DramaType })
}).AsEnumerable();
I've made this much faster by first fetching all recent dramas and then sending a separate query to get the people. It uses a PredicateBuilder.
var dramasByPerson = ctx.Dramas.Where(d => d.DateHappened >= startDate)
.Select(d => new { d.PersonId, d.Id, d.DramaType })
.ToLookup(d => d.PersonId);
var predicate = dramasByPerson.Select(o => o.Key)
.Aggregate(
PredicateBuilder.False<Person>(),
(current, personId) => current.Or(o => o.PersonId == personId)
);
var dictPeople = ctx.People.Where(predicate)
.Select(o => new { o.PersonId, o.LastName, o.FirstName })
.ToDictionary(o => o.PersonId);
var people = dramasByPerson.Select(o => new {
LastName = people[o.Key].LastName,
FirstName = people[o.Key].FirstName,
Dramas = o.Select(d => new { d.Id, d.DramaType })
});