Develop Reference

How do I get the most recent entry by condition in EF Core?

I have a table with the following structure (and sample data):
Identifier
UseDate
PartId
a123
05/01/2000
237
a123
05/01/2000
4656
a123
01/01/2000
2134
a124
04/01/2000
5234
a124
01/01/2000
2890
I need to get the most recent entry of every (non-unique) identifier, but at most one per identifier.
The SQL-Query (MariaDB) that seems to fulfill my problem is the following:
SELECT a.Identifier, a.MaxDate, b.PartId, b.UseDate
FROM
(SELECT Identifier, MAX(UseDate) AS MaxDate FROM MyTable GROUP BY Identifier) a
LEFT JOIN MyTable b ON a.Identifier = b.Identifier
WHERE a.MaxDate = b.UseDate GROUP BY a.Identifier;
However I need this to work with C# and EF Core (Pomelo.EntitiFrameworkCore.MySql 5.0.3), my attempts have been:
var q1 = db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => new { Identifier = t.Key, MaxDate = t.Max(x => x.UseDate) });
return new ObjectResult(db.MyTable
.Join(
q1,
t1 => t1.Identifier,
t2 => t2.Identifier,
(t1, t2) => new { Identifier = t2.Identifier, PartId = t1.PartId, MaxDate = t1.MaxDate, UseDate = t1.UseDate })
.Where(t => t.UseDate == q1.First(x => x.Identifier == t.Identifier).MaxDate)
.GroupBy(t => t.Identifier)
.ToList()
);
and
return new ObjectResult(db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => t.OrderByDescending(x => x.UseDate).FirstOrDefault())
.ToList()
);
The first one throws this error:
System.InvalidOperationException: "Unable to translate the given 'GroupBy' pattern. Call 'AsEnumerable' before 'GroupBy' to evaluate it client-side."
The second one essentially yields the same, just complaining about the LINQ expression instead of the GroupBy.
I want to avoid using raw SQL, but how do I correctly (and hopefully efficiently) implement this?

There are many ways to write such query in LINQ, with most of them being able to be translated by EF Core 5/6+.
The straightforward approach once you have defined a subquery for the necessary grouping and aggregates is to join it to the data table, but not with join operator - instead, use row limiting correlated subquery (SelectMany with Where and Take), e.g.
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => new { Identifier = t.Key, MaxDate = t.Max(x => x.UseDate) })
.SelectMany(g => db.MyTable
.Where(t => t.Identifier == g.Identifier && t.UseDate == g.MaxDate)
.Take(1));
If the ordering field is unique per each other key value (i.e. in your case if UseDate is unique per each unique Identifier value), you can use directly Join operator (since lo limiting is needed), e.g.
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => new { Identifier = t.Key, MaxDate = t.Max(x => x.UseDate) });
.Join(db.MyTable,
g => new { g.Identifier, UseDate = g.MaxDate },
t => new { t.Identifier, t.UseDate },
(g, t) => t);
or directly apply Max based Where condition to the data table:
var query = db.MyTable
.Where(t => t.UseDate == db.MyTable
.Where(t2 => t2.Identifier == t.Identifier)
.Max(t2 => t2.UseDate)
);
Finally, the "standard" LINQ way of getting top 1 item per group.
For EF Core 6.0+:
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(g => g
.OrderByDescending(t => t.UseDate)
.First());
For EF Core 5.0 the grouping result set inside the query must be emulated:
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(g => db.MyTable
.Where(t => t.Identifier == g.Key)
.OrderByDescending(t => t.UseDate)
.First());

elasticsearch NEST get nested document

situation is this. I have in elastic a group. each of these groups have a nested list of items.
Both group and items have an attribute named serial, which are unique.
I get a serial for the group and a serial for item, and with those 2 items i'm supposed to return the item.
Currently i'm doing it the following way:
public item findItem(string groupSerial, string itemSerial)
{
var searchResponse = _elasticClient.Search<Group>(s => s
.Index(_config.groupIndexName)
.Query(q => q
.ConstantScore(cs => cs
.Filter(f => f
.Term(t => t
.Field(fi => fi.serial)
.Value(groupSerial)
)
)
)
).Query(q => q
.Nested(c => c
.InnerHits(i => i.Explain())
.Path(p => p.items)
.Query(nq => nq.Term(t => t
.Field(field => field.items.First().serial)
.Value(itemSerial)))))
);
var result = searchResponse.Documents.FirstOrDefault();
return result?.items.Find(item => item.serial == itemSerial);
}
I get the feeling that there is supposed to be a more efficient way. Like getting the item straight from the search in elastic. Does anyone know how?

Casting Nhibernate result into IDictionary<string,int>

I am trying to convert the result of the query into IDictionary
Here string will contain orderId and the int will contain the TradedQuantity
The query below should join three objects Order, OrderRevision and OrderEvent.
1 Order can have many orderRevisions
1 OrderRevision can have many orderEvents
What the query is trying to do is to inner join three objects and get all order objects whose order id matches the list of orderids supplied to it. Then it does a group by based on orderId and gets the latest TradedQuantity from orderEvents object. LatestTradedQuantity will be the TradedQuantityFrom latest OrderEvent. For now the latest orderevent can be regarded as the one that has highest OrderEventId value.
OrderRevision revisionAlias = null;
Order orderAlias = null;
var query =
Session.QueryOver<OrderEvent>()
.JoinAlias(oe => oe.OrderRevision,() => revisionAlias)
.JoinAlias(oe => oe.OrderRevision.Order,() => orderAlias)
.Where(x => x.OrderRevision.Order.SourceSystem.Name.ToLower() == sourceSystem.ToLower())
.WhereRestrictionOn(x => x.OrderRevision.Order.Id).IsIn(orderIds.ToList())
.SelectList(list => list.SelectGroup(x => x.OrderRevision.Order.SourceOrderIdentifier)
.SelectMax(x => x.Id).Select(x => x.TradedQuantity))
.Select(x => new KeyValuePair<string, int?>(x.OrderRevision.Order.SourceOrderIdentifier, x.TradedQuantity)
);
As this query does not do what is supposed to. Could you please help and let me know how the result can be cast into IDictionary?

You have tagged your question with linq-to-nhibernate, so I guess using it instead of queryover would suit you. With Linq, use a sub-query for selecting the "max" order events ids for each order, then query them and project them to a dictionary.
using System.Linq;
using NHibernate.Linq;
...
var orderEventsIdsQuery = Session.Query<OrderEvent>()
.Where(oe => orderIds.Contains(oe.OrderRevision.Order.Id))
.GroupBy(oe => oe.OrderRevision.Order.SourceOrderIdentifier,
(soi, oes) => oes.Max(oe => oe.Id));
var result = Session.Query<OrderEvent>()
.Where(oe => orderEventsIdsQuery.Contains(oe.Id))
.ToDictionary(oe => oe.OrderRevision.Order.SourceOrderIdentifier,
oe => oe.TradedQuantity);
This should do the job. I do not use QueryOver and I will not try to give an answer for doing it with QueryOver.

How Can I Generate Nhibernate GROUP BY without SELECT the property

I'd like to resolve that problem :
SELECT Max(Date)
FROM Table
GROUP BY SubId
(Then pass it as a SubQuery to mid-action so I can get the Id of the item in Table)
SELECT Id
FROM Table
WHERE Date in
[[[ previous request ]]]
(Then Get the full Table Item with other table join)
SELECT *
FROM Table
LEFT JOIN...
WHERE Id in
[[[ previous request ]]]
I tried this kind of request :
var subquery = QueryOver.Of<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id)
);
var filter = QueryOver.Of<Table>().WithSubquery.
WhereExists(subquery)
.Select(p => p.Id);
var result = Session.QueryOver<Table>().WithSubquery.WhereProperty(p => p.Id).In(filter).Left.JoinQueryOver(p => p.Sub).List();
But the problem is that I can't get the first request right with only the date out of my request.
Is there a better way to do it than that kind of subqueries ? And is there a possibility in NHibernate to Groupy By a Property without selecting it ?
Thanks !

Finally did it that way and it generated the SQL i wanted. But it wasn't 3 subqueries exactly it was 3 queries looking in a set of datas (The arrays subquery and CorrespondingIds).
var subquery = Session.QueryOver<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id))
.List<object[]>().Select(p => p[0]).ToArray();
var CorrespondingIds = Session.QueryOver<Table>(() => x)
.WhereRestrictionOn(() => x.Date).IsIn(subquery)
.Select(p => p.Id).List<int>().ToArray();
var result = Session.QueryOver<Table>(() => x).WhereRestrictionOn(() => x.Id).IsIn(CorrespondingIds).Left.JoinQueryOver(p => p.Sub).List();

Multi Terms search NEST C#

I want to do a search matching multiple values ( an array of values ) like this :
var result1 = _client.Search<type1>(s => s
.Fields(f => f.trip_id)
.Query(q => q
.Terms(t => t.arg1, value1)).Take(_allData))
.Documents.Select(d => d.arg2).ToArray();
var result2 = _client.Search<type2>(s => s
.Query(q => q
.Terms(t => t.arg3, result1))
.Take(_allData)
).Documents.Select(s => s.ar3).ToList();
How can I do ? I was thinking about facets but I don't see how I can do it.
The only way for now that works is with a foreach iterator which is not really effective...
Thanks for your help.

You can express multiple queries like so:
.Query(q=>q.Terms(t=>t.arg3, result1) && q.Terms(t=>t.arg1, value1))
Be sure to read the documentation on writing queries to discover all the good stuff NEST has to offer.

Orelus,
I'd like to use your solution with
.And( af=>af.Term(...), af=>af.Term(...) )
I don't understand where this fits, here's an example of my non-working filter
var results = client.Search<music>(s => s
.Query(q => q
.Filtered(f => f.
Filter(b => b.Bool(m => m.Must(
t => t
.Term(p => p.artist, artist)
&& t.Term(p2 => p2.year, year)
)
)
)
)
)
);