I have a catalog of products that I want to calculate aggregates on. This is simple enough for top level properties such as brand name, manufacturer, etc. The trouble comes with trying to calculate range counts on prices because we sell in multiple currencies, and when determining these counts I only want to query on one currency at a time. Here is a sample of my product object mapping:
public class Product
{
public int ID { get; set;}
public string Name { get; set; }
public IList<Price> Prices { get; set; }
}
public class Price
{
public int CurrencyID { get; set; }
public decimal Cost { get; set; }
}
Here is an example of a query for all products with a price below 100:
var cheapProducts = client.Search<Product>(s => s
.From(0)
.Size(1000)
.Query(q => q
.Range(r => r
.LowerOrEquals(100)
.OnField(f => f.Prices.FirstOrDefault().Cost))));
The ElasticSearch request that this generates is:
{
"from": 0,
"size": 1000,
"query": {
"range" : {
"prices.cost": {
"lte": "100"
}
}
}
}
This returns all products with at least one price below 100 in any currency, as you would expect. What I've been unable to do is to run this query against just prices in a given currency. For example, adding this filter to the query only removes products that don't have a price in currency 1:
var cheapProducts = client.Search<Product>(s => s
.From(0)
.Size(1000)
.Filter(f => f
.Term(t => t
.Prices.FirstOrDefault().CurrencyID, 1))
.Query(q => q
.Range(r => r
.LowerOrEquals(100)
.OnField(f => f.Prices.FirstOrDefault().Cost))));
I've tried treating the Prices list as both a nested object and a child object, but ElasticSearch doesn't appear to be indexing the prices in that way because I get an error of "AggregationExecutionException[[nested] nested path [prices] is not nested]" and similar for HasChild queries. Is it possible to generate queries and aggregates in this way?
First you need to map the nested type:
public class Product
{
public int ID { get; set; }
public string Name { get; set; }
[ElasticProperty(Type = FieldType.Nested)]
public IList<Price> Prices { get; set; }
}
After that, try execute this query:
var cheapProducts = client.Search<Product>(s => s
.From(0)
.Size(1000)
.Query(x => x.Term(p => p
.Prices.First().CurrencyID, 1) && x.Range(r => r
.LowerOrEquals(100)
.OnField(f => f.Prices.FirstOrDefault().Cost))));
Related
I have the following entities and a database context
public class Item
{
public int Id { get; set; }
public int? ReceiptId { get; set; }
public int ItemTypeId { get; set; }
}
public class ItemType
{
public int Id { get; set; }
public string Name { get; set; }
public List<Item> Items { get; set; }
}
public class Receipt
{
public int Id { get; set; }
public string ReceiptInfo { get; set; }
public List<Item> Items { get; set; }
}
I'm trying to get a the list of receipts, but instead of containing the items they contain, I want them to have the itemType and the amount of items for each. I have written the following linq query, which works:
var result = _databaseContext.Receipts.Select(r => new
{
r.Id,
r.ReceiptInfo,
ItemInfo = r.Items.GroupBy(item => item.ItemTypeId)
.Select(group => new
{
IdItemType = group.Key,
AmountOfItems = group.Count(),
}).ToList()
});
With EF Core 7, it is translated to the following SQL query:
SELECT [r].[Id], [r].[ReceiptInfo], [t].[IdItemType], [t].[AmountOfItems]
FROM [Receipts] AS [r]
OUTER APPLY
(SELECT [i].[ItemTypeId] AS [IdItemType], COUNT(*) AS [AmountOfItems]
FROM [Items] AS [i]
WHERE [r].[Id] = [i].[ReceiptId]
GROUP BY [i].[ItemTypeId]) AS [t]
ORDER BY [r].[Id]
Yet, I need to do this in an older project which doesn't support a version older than 3.1 for EF Core.
There it translates the query differently and I get this error
Column 'Receipts.Id' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause
In case of EF Core 3.1, you have to postprocess loaded detail items on the client side:
var rawData = _databaseContext.Receipts.Select(r => new
{
r.Id,
r.ReceiptInfo,
RawItemInfo = r.Items.Select(item => new
{
IdItemType = item.ItemTypeId
}).ToList()
})
.ToList();
var result = rawData
.Select(r => new
{
r.Id,
r.ReceiptInfo,
ItemInfo = r.RawItemInfo
.GroupBy(item => item.ItemTypeId)
.Select(group => new
{
IdItemType = group.Key,
AmountOfItems = group.Count(),
}).ToList()
});
As you see, GroupBy support has improved drastically in EFC 7. EFC 3 only supports GroupBy with aggregates at the query root.
Therefore, to make it run in EFC 3 you need to force the query into the supported shape. To get the same grouping level, the query starts at Items and groups + aggregates once over three elements instead of two:
var result = _databaseContext.Items
.GroupBy(item => new { item.ReceiptId, item.Receipt.ReceiptInfo, item.ItemTypeId })
.Select(group => new
{
Id = group.Key.ReceiptId,
ReceiptInfo = group.Key.ReceiptInfo,
IdItemType = group.Key.ItemTypeId,
NrOfItems = group.Count(),
})
That returns the same data as the original query and does the reduction of data (aggregate) in the database. To get the same result shape, it needs some post-processing in-memory (i.e. after calling AsEnumerable()):
.AsEnumerable()
.GroupBy(x => new { x.Id, x.Receipt.ReceiptInfo })
.Select(g => new
{
g.Key.Id,
g.Key.ReceiptInfo,
ItemInfo = g.Select(x => new { x.IdItemType, x.NrOfItems })
});
This requires adding a navigation property Item.Receipt.
I've been struggling for a few days to display something like Pivoting a dynamic table on SQL, using Linq. I don't actually want to create a pivoted table, I just want to display the results as if it was a pivoted table.
I have two entities:
Category:
public int Id { get; set; }
public string Name { get; set; }
public string Icon { get; set; }
public ICollection<StaticEvent> StaticEvents { get; set; }
StaticEvent:
public int Id { get; set; }
public int CategoryId {get; set; }
public Category Category { get; set; }
public DateTimeOffset Time {get; set; }
[MaxLength(500)]
public string Comment {get; set;}
I'm trying to generate a table showing the COUNT of STATIC EVENTS per CATEGORY PER YEARMONTH. So the rows would be the YEARMONTH, the columns would be the Categories and the values would be the COUNT.
I got to the point where I can count the STATIC EVENTS per YEARMONTH:
var query = _context.Categories
.SelectMany(c => c.StaticEvents )
.GroupBy(c =>
new {
Year = c.Time.Year,
Month = c.Time.Month
})
.Select( x=> new {YearMonth = x.Key, Count = x.Count()})
.ToList();
foreach (var x in query)
{Console.WriteLine($"Month = {x.YearMonth.Year},{x.YearMonth.Month}, , Count = " + x.Count);}
but I'm lost about what to do from here.
If Categories is unknown values which are stored in database, it is not possible to do that via LINQ without dynamic query creation.
There is query which do the job when you known categories on the compile time. Since you have not specified EF version, I have emulated Count with Sum:
var query = _context.Categories
.SelectMany(c => c.StaticEvents)
.GroupBy(c =>
new {
Year = c.Time.Year,
Month = c.Time.Month
})
.Select(x => new {
YearMonth = x.Key,
Cat1 = x.Sum(z => z.CategoryId == 1 ? 1 : 0),
Cat2 = x.Sum(z => z.CategoryId == 2 ? 1 : 0),
Cat3 = x.Sum(z => z.CategoryId == 3 ? 1 : 0),
})
.ToList();
I'm fetching Invoices from database and I want to return all invoices without grouping them!
I don't want to group them since If there are 100 invoices I want to return all of them 100, considering that I want to get Sum of Amount.
So it is perfectly fine to repeat same Total for multiple invoices if their sum is the same, so basically I want to calculate sum of Amount of each invoice item and group by CompanyId, PackageId, BankId, PayMethod only if it's possible?
-- Read code comments --
var result = await _dbContext.Invoices
.Where(p => p.CheckDate >= startDate && p.CheckDate <= endDate)
.Select(p => new DemoDto()
{
CompanyId = p.CompanyId,
Title = p.Title,
Price = p.Price
Total = p.Sum(p => p.Amount).ToString(), // Can I sum here and group by fields I mentioned above? without grouping all data set because I want to keep all 100 records if I received all 100 from database
})
.ToListAsync();
This query obliviously doesn't work because it says
Invoice does not contain definition for Sum and no accessible method..
DemoDto looks like this:
public class DemoDto
{
public string CompanyId {get;set;}
public string Title {get;set;}
public decimal Price {get;set;}
public decimal Amount {get;set;}
}
Invoice class looks like this:
public class Invoice
{
public string CompanyId { get; set; }
public int PackageId {get; set;}
public int BankId {get;set;}
public int PayMethod {get;set;}
public string Title { get; set; }
public decimal Price { get; set; }
public decimal Amount { get; set; }
}
what I'm missing here?
How can I achieve this?
Thanks guys
Cheers
Fetch all the invoices from the database:
var invoices = await _dbContext.Invoices
.Where(p => p.CheckDate >= startDate && p.CheckDate <= endDate)
.ToListAsync();
Group the in-memory results using Linq-To-Object:
var result = invoices?
.GroupBy(p => new { p.CompanyId, p.PackageId, p.BankId, p.PayMethod })
.SelectMany(x => x.Select(y =>
new DemoDto
{
CompanyId = y.CompanyId,
Title = y.Title,
Price = y.Price,
Total = x.Sum(z => z.Price)
}))
.ToList();
If you want to perform the grouping in the database for some reason, you should execute a raw SQL query or a stored procedure rather than relying on the ORM to generate some magic (and most probably inefficient) query for you.
As an exercise in learning EF, I have the following 4 tables. Person 1toM, with Orders M2M, with Products via OrderProducts (Gender is an Enum):
public class Person
{
public int Id { get; set; }
public string Name { get; set; }
public Gender Gender { get; set; }
public IList<Order> Orders { get; set; }
}
public class Order
{
public int Id { get; set; }
public int PersonId { get; set; }
public Person Person { get; set; }
public IList<OrderProduct> OrderProducts { get; set; }
}
public class OrderProduct
{
public int Id { get; set; }
public int Qty { get; set; }
public Order Order { get; set; }
public int OrderId { get; set; }
public Product Product { get; set; }
public int ProductId { get; set; }
}
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public decimal Price { get; set; }
public IList<OrderProduct> OrderProducts { get; set; }
}
I'm trying to generate the following with LINQ extension methods:
A list of products and total [=Sum(OrderProducts.Qty * Product.Price)] spent, grouped by product name then grouped by gender.
The result would look something like this:
Female
Ball $655.60
Bat $1,925.40
Glasses $518.31
Etc...
Male
Ball $1,892.30
Bat $3,947.07
Glasses $1,315.71
Etc...
I'm committing myself to LINQ extension methods and hope that I can also develop some best practice here. I can't work out how to now group by ProductName and aggregate the Qty * Price into a total by product:
var prodsalesbygender = context.Orders
.GroupBy(o => new
{
Gender = o.Person.Gender
})
.Select(g => new
{
ProductName = g.Select(o => o.OrderProducts.Select(op => op.Product.Name)),
Qty = g.Select(o => o.OrderProducts.Select(op => op.Qty)),
Price = g.Select(o => o.OrderProducts.Select(op => op.Product.Price))
}
);
I've tried adding .GroupBy(g => g.ProductName) to the end but get the error "The key selector type for the call to the 'GroupBy' method is not comparable in the underlying store provider.".
I think you are almost there..
Try this one:
var prodsalesbygender = context.Orders
.GroupBy(o => new
{
Gender = o.Person.Gender
})
.Select(g => new
{
Gender = g.Key,
Products = g.Select(o => o.OrderProducts
.GroupBy(op => op.Product)
.Select(op => new
{
ProductName = op.Key.Name,
Qty = op.Sum(op2 => op2.Qty),
Price = op.Select(x => x.Product.Price)
.First(),
})
.Select(x => new
{
ProducName = x.ProductName,
Qty = x.Qty,
Price = x.Price,
TotalPrice = x.Qty * x.Price
}))
});
In short, you just need more projection. In my suggested solution, first you group by the gender. The next step is to project the gender and 'list of product' directly (yes, the difficult part to get around is the OrderProducts). Within the Products we group it by product name, then take the total quantity (Qty) and set the Price - assuming the price for the same product is constant. The next step is to set the TotalPrice, the Qty * Price thing.
Ps. I am fully aware that this query had many deficiencies. Perhaps LINQ expert can give a better help on this.
It will result in a class something like:
{
Gender Gender
IEnumerable<{ ProductName, Qty, Price, TotalPrice }> Products
}
Yes, so much for anonymous type..
Nevertheless, i am baffled by the down votes as the question contains the models in question and the attempt OP have provided.
Finally here is my solution, producing a result exactly as required.
var prodsalesbygender = context.OrderProducts
.GroupBy(op => new
{
Gender = op.Order.Person.Gender
})
.Select(g => new
{
Gender = g.Key.Gender,
Products = g.GroupBy(op => op.Product)
.Select(a => new
{
ProductName = a.Key.Name,
Total = a.Sum(op => op.Qty * op.Product.Price)
})
.OrderBy(a => a.ProductName)
});
foreach (var x in prodsalesbygender)
{
Console.WriteLine(x.Gender);
foreach (var a in x.Products)
Console.WriteLine($"\t{a.ProductName} - {a.Total}");
}
My thanks to #Bagus Tesa
Trying to get a query to work, but honestly not sure how (or if it's even possible) to go about it as everything I have tried hasn't worked.
Querying a total of 6 tables: Person, PersonVote, PersonCategory, Category, City, and FirstAdminDivision.
PersonVote is a user review table for people and contains a column called Vote that is a decimal accepting a value from 1-5 (5 being "best"). FirstAdminDivision would be synonymous with US states, like California. Person table has a column called CityId which is the foreign key to City. The other tables I believe are mostly self-explanatory so I won't comment unless needed.
My goal is create a query that returns a list of the "most popular" people which would be based on the average of all votes on the PersonVote table for a particular person. For instance, if a person has 3 votes and all 3 votes are "5" then they would be first in the list...don't really care about secondary ordering at this point...eg...like most votes in a tie would "win".
I have this working without AutoMapper, but I love AM's ability to do projection using the ProjectTo extension method as the code is very clean and readable and would prefer to use that approach if possible but haven't had any luck getting it to work.
Here is what I have that does work....so basically, I am trying to see if this is possible with ProjectTo instead of LINQ's Select method.
List<PersonModel> people = db.People
.GroupBy(x => x.PersonId)
.Select(x => new PersonModel
{
PersonId = x.FirstOrDefault().PersonId,
Name = x.FirstOrDefault().Name,
LocationDisplay = x.FirstOrDefault().City.Name + ", " + x.FirstOrDefault().City.FirstAdminDivision.Name,
AverageVote = x.FirstOrDefault().PersonVotes.Average(y => y.Vote),
Categories = x.FirstOrDefault().PersonCategories.Select(y => new CategoryModel
{
CategoryId = y.CategoryId,
Name = y.Category.Name
}).ToList()
})
.OrderByDescending(x => x.AverageVote)
.ToList();
By looking at your code sample I tried to determine what your models would be in order to setup an example. I only implemented using a few of the properties to show the functionality:
public class People
{
public int PeronId { get; set; }
public string Name { get; set; }
public City City { get; set; }
public IList<PersonVotes> PersonVoes { get; set; }
}
public class PersonVotes
{
public int Vote { get; set; }
}
public class City
{
public string Name { get; set; }
}
public class FirstAdminDivision
{
public string Name { get; set; }
}
public class PersonModel
{
public int PersonId { get; set; }
public string Name { get; set; }
public string LocationDisplay { get; set; }
public double AverageVote { get; set; }
}
To use the ProjectTo extension I then initialize AM through the static API:
Mapper.Initialize(cfg =>
{
cfg.CreateMap<IEnumerable<People>, PersonModel>()
.ForMember(
model => model.LocationDisplay,
conf => conf.MapFrom(p => p.FirstOrDefault().City.Name))
.ForMember(
model => model.AverageVote,
conf => conf.MapFrom(p => p.FirstOrDefault().PersonVoes.Average(votes => votes.Vote)));
});
So given the following object:
var people = new List<People>()
{
new People
{
PeronId = 1,
City = new City
{
Name = "XXXX"
},
PersonVoes = new List<PersonVotes>
{
new PersonVotes
{
Vote = 4
},
new PersonVotes
{
Vote = 3
}
}
}
};
I would then a have query:
var result = people
.GroupBy(p => p.PeronId)
.Select(peoples => peoples)
.AsQueryable()
.ProjectTo<PersonModel>();
I'm just using in memory objects so that is why I convert to IQueryable to use the ProjectTo extension method in AM.
I'm hoping this was what you're looking for. Cheers,
UPDATED FOR LINQ TO ENTITIES QUERY:
var result = db.People
.GroupBy(p => p.PersonId)
.ProjectTo<PersonModel>(base.ConfigProvider) // AM requires me to pass Mapping Provider here.
.OrderByDescending(x => x.AverageVote)
.ToList();