After some hours of trying and researching I'm still stuck with the following problem.
I have a list of customers, each customer has a list of orders. I now want to sort the list of customers by their highest order number.
Another aspect is that not every customer has an order, so for sorting i want to replace the missing order number with an “-“ sign.
My first approach won’t work because it can’t compare the object, but it explains the problem quite well.
customers.OrderBy(m => m.Orders == null ? "-" : m.Orders.LastOrDefault().OrderNumber.ToString());
After reading the link below I ended up with this code, but that is not working either.
how do I sort a collection with child property?
customers.OrderBy(c => c.Orders.OrderBy(o => o == null ? "-" : o.OrderNumber.ToString()).Select(o => o == null ? "-" : o.OrderNumber.ToString()).ToList();
Can someone tell me how I can do this best?
Best Regards
Edit:
Here are the entities (not full entity) I'm trying to sort.
I'm working with entity framework and I've already read out the customers as an IQueryable and now I need to sort this.
Customer
public class Customer
{
public int ID { get; set; }
public virtual List<Order> Orders { get; set; }
}
Order
public class Order {
public int ID { get; set; }
public int OrderNumber { get; set; }
public virtual Customer Customer { get; set; }
}
Now the customer does not necessarily have a list of orders associated.
Meaning a customer can have 0,1 or n orders assigned to him, that's what I meant with null.
I now want to sort the list of customers by their highest order number.
Then the following should do the trick:
var result = customers.OrderBy(c => c.Orders.Max(o => (int?)o.OrderNumber));
The int? cast is needed to let Max function return null for customers w/o orders. This normally would put them at the beginning of the sort order.
In case you want to put customers w/o orders at the end, the simplest would be to use int.MaxValue (assuming you have no OrderNumber == int.MaxValue):
var result = customers.OrderBy(c => c.Orders.Max(o => (int?)o.OrderNumber) ?? int.MaxValue);
You could first remove the customers with no Orders and then add them back with Concat()
customers
.Where(c => c.Orders.Count() != 0)
.OrderBy(c => c.Orders.OrderBy(o => o.OrderNumber).Select(o => o.OrderNumber).FirstOrDefault())
.Concat(customers.Where(c => c.Orders.Count() == 0))
.ToList();
Let me know if that works for your use-case.
So Long
Related
I have a domain class like below :
public class Employee
{
public int EmployeeId { get; set; }
public int DeptId { get; set; }
}
public class Transaction
{
public int TRID { get; set; }
public int EmployeeId { get; set; }
public string Status { get; set; }
}
Now I want to get all employees from the EmployeeTable for DeptId = 100. I want to calculate Pending status for those employees whose transactions are pending.
So if employee records are found in Transactions table then just want to return a column saying whether employee has any pending transactions or not)
Query :
var t = (from e in _employeeRepository.GetAll() //returns IQueryable<Employee>
where e.DeptId == 100
from t in _transactionRepository.GetAll().Where(t => t.EmployeeId == e.EmployeeId)
select new
{
IsPendingTransaction = (t != null && t.Status != "Done") ? true : false,
}).ToList();
Error : LINQ to Entities does not recognize the method
'System.Linq.IQueryable`1[Transaction] GetAll()' method, and this
method cannot be translated into a store expression."}
Sql Query :
SELECT e.*
(CASE WHEN (t.EmployeeId is not null and t.Status <> 'Done')
THEN CAST(1 AS BIT)
ELSE CAST(0 AS BIT)
End) as IsPendingTransaction
FROM Employee e OUTER APPLY
(SELECT t.*
FROM Transactions t
WHERE e.EmployeeId = t.EmployeeId
) t
WHERE e.DeptId = 100;
The issue is that when you work within IQueryable, every statement inside that Linq expression must be understood by EF to be able to be translated to SQL.
Your first repository call returns an IQueryable<Employee> which you are trying to extend by telling it to join on some code called "_transactionRepository.GetAll()" EF doesn't know what this is, it doesn't correlate to mapped DbSets or properties on entities...
If your Transaction entity has a navigation property back to Employee (which it should) you should be able to accomplish what you want using just the TransactionRepository with something like:
var t = _transactionRepository.GetAll()
.Where(t => t.Employee.DeptId == 100)
.Select(t => new
{
IsPendingTransaction = (t != null && t.Status != "Done") ? true : false
}).ToList();
Using IQueryable<TEntity> in a repository pattern can be quite powerful, however I don't recommend adopting a Generic repository pattern as it just serves to fragment your thinking when working with entities and their relationships with one another, allowing EF to manage the resulting SQL without you resorting to pre-emptively trying to do the joining yourself, often causing conflicts with what EF is capable of working out itself.
Edit: Ok, from your description to get a list of employees with a flag if they have a pending transaction: That would be back at the Employee level with a query something like:
var employees = _employeeRepository.GetAll()
.Where(e => e.DeptId == 100)
.Select(e =>
{
Employee = e,
HasPendingTransaction = e.Transactions.Any(t => t.Status != "Done")
}).ToList();
Or projected to a ViewModel to embed the HasPendingTransaction alongside the Employee details:
var employees = _employeeRepository.GetAll()
.Where(e => e.DeptId == 100)
.Select(e => new EmployeeDetailsViewModel
{
EmployeeId = e.EmployeeId,
Name = e.Name,
// include other relevent details needed for the view...
HasPendingTransaction = e.Transactions.Any(t => t.Status != "Done")
}).ToList();
The advantage of projection is you can build more efficient / faster queries that reduce the amount of data sent over the wire and avoid issues like lazy load trips if you try to serialize entities to the view.
Fix Transaction class
public class Transaction
{
public int TRID { get; set; }
public string Status { get; set; }
public int EmployeeId { get; set; }
public virtual Employee Employee { get; set; }
}
It is not the best idea to have a separate repository for each entity since query usually consists from several entities. It is better to make a join using dbcontext then several repository queries as you trying to do. Don't try to create a base generic repository also. Sooner or later you will see that is is not very helpfull. So add in one of your repositories (probably EmployeeRepository) query like this
var employees= dbcontext.Transactions
.Where(t=> t.Employee.DeptId == 100 && t.EmployeeId==employeeId)
.Select (t=> new {
EmployeeName= t.Employee.Name,
IsPendingTransaction = (t.Status != null && t.Status != "Done") ? true : false}).ToList()
This question already has answers here:
LINQ To Entities does not recognize the method Last. Really?
(6 answers)
Closed 3 years ago.
Is there a way to order, let's say, all customers by the date of the last purchase?
For example
ctx.Customers.OrderByDescending(e => e.Purchases.LastOrDefault().DateTime);
It would be something like this, however, this doesn't work. It throws the exception
LINQ to Entities does not recognize the method 'Purchase
LastOrDefault[Purchase]
(System.Collections.Generic.IEnumerable`1[Purchase])'
method, and this method cannot be translated into a store expression
edit:
public class Customer
{
public Customer()
{
Purchases = new List<Purchase>();
}
public int Id { get; set; }
public string Name { get; set; }
[JsonIgnore]
public virtual IList<Purchase> Purchases { get; set; }
}
public class Purchase
{
public int Id { get; set; }
public int IdCustomer { get; set; }
public DateTime DateTime { get; set; }
public virtual Customer Customer { get; set; }
}
In Context I do have somthing like
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.HasRequired(s => s.Customer)
.WithMany(p => p.Purchases)
.HasForeignKey(s => s.IdCustomer);
}
ctx.Customers.OrderByDescending(e => e.Purchases.LastOrDefault().DateTime);
looks like a context query (Entity Framework, usually dbContext), so here you have an IQueryable not a List.
Entity Framework will try to convert this to a SQL statement before giving you results, but
SELECT * BOTTOM(X) FROM TABLE ORDER BY Purchases desc
is not an expression, but more importantly EF just doesn't recognize what you want to do.
Instead, you just want to flip the logic to:
SELECT * TOP(X) FROM TABLE ORDER BY Purchases asc
Or:
ctx.Customers.OrderBy(e => e.Purchases.FirstOrDefault().DateTime);
or you can order by on your subquery:
ctx.Customers.OrderBy(e => e.Purchases.OrderByDescending(x => x.propertyToSortOn)
.FirstOrDefault().DateTime);
Getting the last n records from the bottom of a sorted list, is actually the same as getting the top n from a list sorted the other way:
1,2,3,4,5,6 -> top 3 in ascending order = 1,2,3
6,5,4,3,2,1 -> bottom 3 in descending order = 3,2,1
LastOrDefault is not supported in Linq-to-Entities (meaning they have not yet developed a way to translate that to the equivalent SQL code). One option is to use AsEnumerable to do the ordering in memory:
ctx.Customers
.AsEnumerable()
.OrderByDescending(e => e.Purchases.LastOrDefault().DateTime);
However, since the order of Purchases is not deterministic, you may want to specify an order there as well:
ctx.Customers
.AsEnumerable()
.OrderByDescending(e => e.Purchases.OrderBy(p => p.DateTime).LastOrDefault());
or just use Max on the `Purchases':
ctx.Customers
.AsEnumerable()
.OrderByDescending(e => e.Purchases.Max(p => p.DateTime));
If the performance of any of those queries is not acceptable, the last resort would be to write the direct SQL and pass that to ctx.Customers.SqlQuery()
I'm having trouble with something that is probably very simple.
In my database I have the following tables:
tblOrder
-----------------
Id
OrderStatusId
tblOrderStatus
-----------------
Id
Name
And I have made the following mappings in my project:
[Class(NameType = typeof(Order), Table = "tblOrder")
public class Order {
[Id(-2, Name = "Id")]
[Generator(-1, Class = "native")]
public virtual long Id { get; set; }
[ManyToOne]
public virtual OrderStatus Status { get; set; }
}
[Class(NameType = typeof(OrderStatus), Table = "tblOrderStatus")]
public class OrderStatus {
[Id(-2, Name = "Id")]
[Generator(-1, Class = "native")]
public virtual long Id { get; set; }
[Property]
public virtual string Name { get; set; }
}
The query should return a IList<OrderSummary>. I want the class OrderSummary to have a property Status where Status is an object with an Id and a Name property. This could be either with a KeyValuePair or of type OrderStatus (whichever is best and works). Fetching the orders is not a problem but adding the OrderStatus as an object with said properties is the part I'm having trouble with.
I also need to return the result of the query as JSON to the client.
OrderSummary should look like this:
public class OrderSummary {
public long Id { get; set; }
public OrderStatus Status { get; set; }
}
In my first version OrderSummary had separate properties for OrderStatusId and OrderStatusName. This works but I'm trying to avoid these separate properties.
I have also tried this with SelectSubQuery but this returns an error because it returns more than one field in a subquery.
----------------------------------- UPDATE -----------------------------
Following Fredy Treboux's advice I changed my query using Eager which result in the following query:
var query = session.QueryOver<OrderStatus>
.Fetch(o => o.Status).Eager
.JoinAlias(o => o.Status, () => statusAlias, JoinType.LeftOuterJoin);
The problem is, I found out, is not selecting the data but how to convert the retrieved Status and assign it to OrderSummary.Status? I have tried the following:
OrderSummary orderAlias = null;
query.SelectList(list => list
.Select(o => o.Id).WithAlias(() => orderAlias.Id)
.Select(() => statusAlias).WithAlias(() => orderAlias.Status)
).TransformUsing(Transformer.AliasToBean<OrderSummary>());
-------------------------------- ANSWER ----------------------------------
As I said in my last edit, the problem does not seem to be the actual selection of OrderStatus but returning it to the client. So I thought it was my lack of knowledge of NHibernate instead it was as simple as adding the [JsonObject] attribute to the OrderStatus class. How silly of me.
I have changed my query to the following:
Order orderAlias = null;
OrderSummary orderSummary = null;
OrderStatus statusAlias = null;
var query = session.QueryOver<Order>(() => orderAlias)
.JoinAlias(() => orderAlias.Status, () => statusAlias, JoinType.LeftOuterJoin);
query = query
.Select(
Projections.ProjectionList()
.Add(Projections.Property(() => orderAlias.Id).WithAlias(() => orderSummary.Id))
.Add(Projections.Property(() => orderAlias.Status).WithAlias(() => orderSummary.Status)
);
Result = query.TransformUsing(Tranformers.AliasToBean<OrderSummary>())
.List<OrderSummary>()
.ToList();
I'm afraid that currently it's not possible. I guess that Nhibernate transformers are not able to construct nested complex properties.
You can return list of tuples and then cast it manually to your entity:
OrderStatus statusAlias = null;
var tuples = Session.QueryOver<Order>()
.JoinQueryOver(x => x.Status, () => statusAlias)
.SelectList(list => list
.Select(x => x.Id)
.Select(x => statusAlias.Id)
.Select(x => statusAlias.Name))
.List<object[]>();
var result = tuples.Select(Convert);
private OrderSummary Convert(object[] item) {
return new OrderSummary
{
Id = (long)item[0],
OrderStatus = new OrderStatus { Id = (long)item[1], Name = (string)item[2] }
};
}
Also if you don't bother about performance much it's possible to fetch a list of you Orders and convert it to OrderSummary. You can do it by simply define casting operator or using some tool like AutoMapper or ExpressMapper.
Sorry I didn't see your comment asking for an example before.
I'm going to leave some code explaining the approach I mentioned, although it was already given as an alternative in the other response and I believe it's the easiest way to go (not using transformers at all):
string GetOrderSummaries()
{
// First, you just query the orders and eager fetch the status.
// The eager fetch is just to avoid a Select N+1 when traversing the returned list.
// With that, we make sure we will execute only one query (it will be a join).
var query = session.QueryOver<Order>()
.Fetch(o => o.Status).Eager;
// This executes your query and creates a list of orders.
var orders = query.List();
// We map these orders to DTOs, here I'm doing it manually.
// Ideally, have one DTO for Order (OrderSummary) and one for OrderStatus (OrderSummaryStatus).
// As mentioned by the other commenter, you can use (for example) AutoMapper to take care of this for you:
var orderSummaries = orders.Select(order => new OrderSummary
{
Id = order.Id,
Status = new OrderSummaryStatus
{
Id = order.Status.Id,
Name = order.Status.Name
}
}).ToList();
// Yes, it is true that this implied that we not only materialized the entities, but then went over the list a second time.
// In most cases I bet this performance implication is negligible (I imagine serializing to Json will possibly be slower than that).
// And code is more terse and possibly more resilient.
// We serialize the DTOs to Json with, for example, Json.NET
var orderSummariesJson = JsonConvert.SerializeObject(orderSummaries);
return orderSummariesJson;
}
Useful links:
AutoMapper: http://automapper.org/
Json.NET: http://www.newtonsoft.com/json
I have 2 different classes that represent 2 types of data. The first is the unposted raw format. The second is the posted format.
public class SalesRecords
{
public long? RecordId { get; set; }
public DateTime RecordDate { get; set; }
public string RecordDesc { get; set; }
// Other non-related properties and methods
}
public class PostedSalesRecords
{
public long? CorrelationId { get; set; }
public DateTime RecordDate { get; set; }
public DateTime? PostedDate { get; set; }
public string RecordDesc { get; set; }
// Other non-related properties and methods
}
Our system has a list of sales records. These sales records are posted to a different system at a time determined by the users. I am creating a screen that will show all of the posted sales records along with the unposted sales records as a reconciliation. The datasource for my grid will be a list of PostedSalesRecords. What I need to do is find out which records out of the List<SalesRecords> that are not in List<PostedSalesRecords> and then map those unposted SalesRecords to a PostedSalesRecords. I am having trouble finding a way to quickly compare. Basically I tried this, and it was EXTREMELY slow:
private List<SalesRecords> GetUnpostedSalesRecords(
List<SalesRecords> allSalesRecords,
List<PostedSalesRecords> postedSalesRecords)
{
return allSalesRecords.Where(x => !(postedSalesRecords.Select(y => y.CorrelationId.Value).Contains(x.RecordId.Value))).ToList();
}
My biggest issue is that I am filtering through a lot of data. I am comparing ~55,000 total sales records to about 17,000 posted records. It takes about 2 minutes for me to process this. Any possible way to speed this up? Thanks!
You can try an outer join, please see if this helps with the performance:
var test = (from s in allSalesRecords
join p in postedSalesRecords on s.RecordId equals p.CorrelationId into joined
from j in joined.DefaultIfEmpty()
where j == null
select s).ToList();
Or in your implementation, you can create a dictionary of only Ids for postedSalesRecords and then use that collection in your query, it'll definitely help with performance because the lookup time will be O(1) instead of traversing through the whole collection for each record.
var postedIds = postedSalesRecords.Select(y => y.CorrelationId.Value)
.Distinct().ToDictionary(d=>d);
return allSalesRecords.Where(x => !(postedIds.ContainsKey(x.RecordId.Value))).ToList();
Using a left outer join as described on MSDN should work much more efficiently:
private List<SalesRecords> GetUnpostedSalesRecords(
List<SalesRecords> allSalesRecords,
List<PostedSalesRecords> postedSalesRecords)
{
return (from x in allSalesRecords
join y in postedSalesRecords on x.RecordId.Value
equals y.CorrelationId.Value into joined
from z in joined.DefaultIfEmpty()
where z == null
select x).ToList();
}
This will probably be implemented with a hash set. You could implement this yourself (arguably clearer that way): build a HashSet<long> of the ID values in one or both lists to ensure that you don't need repetitive O(N) lookups each time you go through the outer list.
Consider these contrived entity objects:
public class Consumer
{
public int Id { get; set; }
public string Name { get; set; }
public bool NeedsProcessed { get; set; }
public virtual IList<Purchase> Purchases { get; set; } //virtual so EF can lazy-load
}
public class Purchase
{
public int Id { get; set; }
public decimal TotalCost { get; set; }
public int ConsumerId { get; set; }
}
Now let's say I want to run this code:
var consumers = Consumers.Where(consumer => consumer.NeedsProcessed);
//assume that ProcessConsumers accesses the Consumer.Purchases property
SomeExternalServiceICannotModify.ProcessConsumers(consumers);
By default this will suffer from Select N+1 inside the ProcessConsumers method. It will trigger a query when it enumerates the consumers, then it'll grab each purchases collection 1 by 1. The standard solution to this problem would be to add an include:
var consumers = Consumers.Include("Purchases").Where(consumer => consumer.NeedsProcessed);
//assume that ProcessConsumers accesses the Consumer.Purchases property
SomeExternalServiceICannotModify.ProcessConsumers(consumers);
That works fine in many cases, but in some complex cases, an include can utterly destroy performance by orders of magnitude. Is it possible to do something like this:
Grab my consumers, var consumers = _entityContext.Consumers.Where(...).ToList()
Grab my purchases, var purchases = _entityContext.Purchases.Where(...).ToList()
Hydrate the consumer.Purchases collections manually from the purchases I already loaded into memory. Then when I pass it to ProcessConsumers it won't trigger more db queries.
I'm not sure how to do #3. If you try to access any consumer.Purchases collection that'll trigger the lazy load (and thus the Select N+1). Perhaps I need to cast the Consumers to the proper type (instead of the EF proxy type) and then load the collection? Something like this:
foreach (var consumer in Consumers)
{
//since the EF proxy overrides the Purchases property, this doesn't really work, I'm trying to figure out what would
((Consumer)consumer).Purchases = purchases.Where(x => x.ConsumerId = consumer.ConsumerId).ToList();
}
EDIT:
I have re-written the example a bit to hopefully reveal the issue more clearly.
If I'm understanding correctly, you would like to load both a filtered subset of Consumers each with a filtered subset of their Purchases in 1 query. If that's not correct, please forgive my understanding of your intent. If that is correct, you could do something like:
var consumersAndPurchases = db.Consumers.Where(...)
.Select(c => new {
Consumer = c,
RelevantPurchases = c.Purchases.Where(...)
})
.AsNoTracking()
.ToList(); // loads in 1 query
// this should be OK because we did AsNoTracking()
consumersAndPurchases.ForEach(t => t.Consumer.Purchases = t.RelevantPurchases);
CannotModify.Process(consumersAndPurchases.Select(t => t.Consumer));
Note that this WON'T work if the Process function is expecting to modify the consumer object and then commit those changes back to the database.
Grab my consumers
var consumers = _entityContext.Consumers
.Where(consumer => consumer.Id > 1000)
.ToList();
Grab my purchases
var purchases = consumers.Select(x => new {
Id = x.Id,
IList<Purchases> Purchases = x.Purchases
})
.ToList()
.GroupBy(x => x.Id)
.Select( x => x.Aggregate((merged, next) => merged.Merge(next)))
.ToList();
Hydrate the consumer.Purchases collections manually from the
purchases I already loaded into memory.
for(int i = 0; i < costumers.Lenght; i++)
costumers[i].Purchases = purchases[i];
Would it not be possible for you to work around the many-roundtrips-or-inefficient-query-generation problem by doing the work on the database - essentially by returning a projection instead of a particular entity, as demonstrated below:
var query = from c in db.Consumers
where c.Id > 1000
select new { Consumer = c, Total = c.Purchases.Sum( p => p.TotalCost ) };
var total = query.Sum( cp => cp.Total );
I'm not an EF expert by any means, so forgive me if this technique is not appropriate.
EF will populate the consumer.Purchases collections for you, if you use the same context to fetch both collections:
List<Consumer> consumers = null;
using ( var ctx = new XXXEntities() )
{
consumers = ctx.Consumers.Where( ... ).ToList();
// EF will populate consumers.Purchases when it loads these objects
ctx.Purchases.Where( ... ).ToList();
}
// the Purchase objects are now in the consumer.Purchases collections
var sum = consumers.Sum( c => c.Purchases.Sum( p => p.TotalCost ) );
EDIT :
This results in just 2 db calls: 1 to get the collection of Consumers and 1 to get the collection of Purchases.
EF will look at each Purchase record returned and look up the corresponding Consumer record from Purchase.ConsumerId. It will then add the Purchase object to the Consumer.Purchases collection for you.
Option 2:
If there is some reason you want to fetch two lists from different contexts and then link them, I would add another property to the Consumer class:
partial class Consumer
{
public List<Purchase> UI_Purchases { get; set; }
}
You can then set this property from the Purchases collection and use it in your UI.