Develop Reference

Nhibernate migrate ICriteria to QueryOver

We are using ICriteria and now we would like to switch to more readable QueryOver in Nhibernate
Can someone give me a hint how to convert this generic pagination logic for Icriteria to QueryOver?
public static PagedList<T> PagedList<T>(this ICriteria criteria,
ISession session, int pageIndex, int pageSize) where T : class
{
if (pageIndex < 0)
pageIndex = 0;
var countCrit = (ICriteria)criteria.Clone();
countCrit.ClearOrders(); // so we don’t have missing group by exceptions
var results = session.CreateMultiCriteria()
.Add<long>(countCrit.SetProjection(Projections.RowCountInt64()))
.Add<T>(criteria.SetFirstResult(pageIndex * pageSize).SetMaxResults(pageSize))
.List();
var totalCount = ((IList<long>)results[0])[0];
return new PagedList<T>((IList<T>)results[1], totalCount, pageIndex, pageSize);
}

The way I am using it:
var session = ... // get a ISession
// the QueryOver
var query = session.QueryOver<MyEntity>();
// apply all filtering, sorting...
query...
// GET A ROW COUNT query (ICriteria)
var rowCount = CriteriaTransformer.TransformToRowCount(query.UnderlyingCriteria);
// ask for a list, but with a Future, to combine both in one SQL statement
var list = query
.Future<MyEntity>()
.ToList();
// execute the main and count query at once
var count = rowCount
.FutureValue<int>()
.Value;
// list is now in memory, ready to be used
var list = futureList
.ToList();
So, we are using QueryOver, and profiting from underlying criteria and transformer. With a Future, we also execute that all at one command.

How to add dynamic sort by direction to NHibernate queryover

I am trying to add dynamic order by direction to my nhibernate queryover. Can anyone help how to do this? I was able to add dynamic orderby field. but dont know how to do the order by direction. Please find below my code:
if (!string.IsNullOrEmpty(sortField))
{
var sortByProperty = Helper.GetSortByProperty(sortField);
if (sortByProperty != null)
{
query.OrderBy(x => sortByProperty.GetValue(x, null));
}
}
var result = query.Skip(pageIndex*pageSize)
.Take(pageSize)
.Future<Member>();

The way I have done this is, passing in a ListSortDirection type variable to the function that is doing the query.
So then you could apply the
OrderBy() which:
Sorts the elements of a sequence in ascending order according to a key.
OrderByDescending() which:
Sorts the elements of a sequence in descending order according to a key.
on your IQueryable<T> depending on the values from ListSortDirection.Ascending or ListSortDirection.Descending.
As requested by OP in comments, added sample generic code:
public IList<T> GetEntityList<T>(Expression<Func<T, bool>> whereExpression, Expression<Func<T, object>> orderByExpression, ListSortDirection orderDirection, int totalPages, int start, int limit)
{
IList<T> returnVar;
using (var session = _nhibernate.OpenSession())
{
var firstQueryable = session.Query<T>().Where(whereExpression);
IQueryable<T> secondQueryable = orderDirection == ListSortDirection.Ascending ? firstQueryable.OrderBy(orderByExpression) : firstQueryable.OrderByDescending(orderByExpression);
returnVar = totalPages > 0 ? secondQueryable.Skip(start).Take(limit).ToList() : secondQueryable.ToList();
}
return returnVar;
}
To address another question in comment from OP, for the QueryOver API, for instance you can use the .OrderBy(orderByExpression).Desc for reverse sorting.

Join large list of Integers into LINQ Query

I have LINQ query that returns me the following error:
"The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Too many parameters were provided in this RPC request. The maximum is 2100".
All I need is to count all clients that have BirthDate that I have their ID's in list.
My list of client ID's could be huge (millions of records).
Here is the query:
List<int> allClients = GetClientIDs();
int total = context.Clients.Where(x => allClients.Contains(x.ClientID) && x.BirthDate != null).Count();
When the query is rewritten this way,
int total = context
.Clients
.Count(x => allClients.Contains(x.ClientID) && x.BirthDate != null);
it causes the same error.
Also tried to make it in different way and it eats all memory:
List<int> allClients = GetClientIDs();
total = (from x in allClients.AsQueryable()
join y in context.Clients
on x equals y.ClientID
where y.BirthDate != null
select x).Count();

We ran into this same issue at work. The problem is that list.Contains() creates a WHERE column IN (val1, val2, ... valN) statement, so you're limited to how many values you can put in there. What we ended up doing was in fact do it in batches much like you did.
However, I think I can offer you a cleaner and more elegant piece of code to do this with. Here is an extension method that will be added to the other Linq methods you normally use:
public static IEnumerable<IEnumerable<T>> BulkForEach<T>(this IEnumerable<T> list, int size = 1000)
{
for (int index = 0; index < list.Count() / size + 1; index++)
{
IEnumerable<T> returnVal = list.Skip(index * size).Take(size).ToList();
yield return returnVal;
}
}
Then you use it like this:
foreach (var item in list.BulkForEach())
{
// Do logic here. item is an IEnumerable<T> (in your case, int)
}
EDIT
Or, if you prefer, you can make it act like the normal List.ForEach() like this:
public static void BulkForEach<T>(this IEnumerable<T> list, Action<IEnumerable<T>> action, int size = 1000)
{
for (int index = 0; index < list.Count() / size + 1; index++)
{
IEnumerable<T> returnVal = list.Skip(index * size).Take(size).ToList();
action.Invoke(returnVal);
}
}
Used like this:
list.BulkForEach(p => { /* Do logic */ });

Well as Gert Arnold mentioned before, making query in chunks solves the problem, but it looks nasty:
List<int> allClients = GetClientIDs();
int total = 0;
const int sqlLimit = 2000;
int iterations = allClients.Count() / sqlLimit;
for (int i = 0; i <= iterations; i++)
{
List<int> tempList = allClients.Skip(i * sqlLimit).Take(sqlLimit).ToList();
int thisTotal = context.Clients.Count(x => tempList.Contains(x.ClientID) && x.BirthDate != null);
total = total + thisTotal;
}

As has been said above, your query is probably being translated to:
select count(1)
from Clients
where ClientID = #id1 or ClientID = #id2 -- and so on up to the number of ids returned by GetClientIDs.
You will need to change your query such that you aren't passing so many parameters to it.
To see the generated SQL you can set the Clients.Log = Console.Out which will cause it to be written to the debug window when it is executed.
EDIT:
A possible alternative to chunking would be to send the IDs to the server as a delimited string, and create a UDF in your database which can covert that string back to a list.
var clientIds = string.Jon(",", allClients);
var total = (from client in context.Clients
join clientIds in context.udf_SplitString(clientIds)
on client.ClientId equals clientIds.Id
select client).Count();
There are lots of examples on Google for UDFs that split strings.

Another alternative and probably the fastest at query time is to add your numbers from the CSV file into a temporary table in your database and then do a join query.
Doing a query in chunks means a lot of round-trips between your client and database. If the list of IDs you are interested in is static or changes rarely, I recommend the approach of a temporary table.

If you don't mind moving the work from the database to the application server and have the memory, try this.
int total = context.Clients.AsEnumerable().Where(x => allClients.Contains(x.ClientID) && x.BirthDate != null).Count();

Remove OrderBy from an IQueryable<T>

I have a paging API that returns rows a user requests, but only so many at one time, not the entire collection. The API works as designed, but I do have to calculate the total number of records that are available (for proper page calculations). Within the API, I use Linq2Sql and I work a lot with the IQueryable before i finally make my requests. When I go to get the count, I call something like: totalRecordCount = queryable.Count();
The resulting SQL is interesting none the less, but it also adds an unnecessary Order By which makes the query very expensive.
exec sp_executesql N'SELECT COUNT(*) AS [value]
FROM (
SELECT TOP (1) NULL AS [EMPTY]
FROM [dbo].[JournalEventsView] AS [t0]
WHERE [t0].[DataOwnerID] = #p0
ORDER BY [t0].[DataTimeStamp] DESC
) AS [t1]',N'#p0 int',#p0=1
Because I am using the IQueryable, I can manipulate the IQueryable prior to it making it to the SQL server.
My question is, if I already have an IQueryable with a OrderBy in it, is it possible to remove that OrderBy before I call the Count()?
like: totalRecordCount = queryable.NoOrder.Count();
If not, no biggie. I see many questions how to OrderBy, but not any involving removing an OrderBy from the Linq expression.
Thanks!

So, the below code is a spike against an in-memory array. There may be some hurdles to get this working with Entity Framework (or some other arbitrary IQueryProvider implementation). Basically, what we are going to do is visit the expression tree and look for any Ordering method call and simply remove it from the tree. Hope this points you in the right direction.
class Program
{
static void Main(string[] args)
{
var seq = new[] { 1, 3, 5, 7, 9, 2, 4, 6, 8 };
var query = seq.OrderBy(x => x);
Console.WriteLine("Print out in reverse order.");
foreach (var item in query)
{
Console.WriteLine(item);
}
Console.WriteLine("Prints out in original order");
var queryExpression = seq.AsQueryable().OrderBy(x => x).ThenByDescending(x => x).Expression;
var queryDelegate = Expression.Lambda<Func<IEnumerable<int>>>(new OrderByRemover().Visit(queryExpression)).Compile();
foreach (var item in queryDelegate())
{
Console.WriteLine(item);
}
Console.ReadLine();
}
}
public class OrderByRemover : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method.DeclaringType != typeof(Enumerable) && node.Method.DeclaringType != typeof(Queryable))
return base.VisitMethodCall(node);
if (node.Method.Name != "OrderBy" && node.Method.Name != "OrderByDescending" && node.Method.Name != "ThenBy" && node.Method.Name != "ThenByDescending")
return base.VisitMethodCall(node);
//eliminate the method call from the expression tree by returning the object of the call.
return base.Visit(node.Arguments[0]);
}
}

There isn't just an unneeded ORDER BY, there's also a spurious TOP(1).
SELECT TOP (1) NULL AS [EMPTY] ...
That subselect will only return 0 or 1 rows. In fact without the TOP there it wouldn't be legal to have an ORDER BY in a subselect.
The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP or FOR XML is also specified.: SELECT COUNT(*) FROM ( SELECT * FROM Table1 ORDER BY foo )
sqlfiddle
I think you have probably done something wrong in your LINQ. Are you sure you haven't written .Take(1) or similar somewhere in your query, before calling .Count()?
This is wrong:
IQueryable<Foo> foo = (...).OrderBy(x => x.Foo).Take(1);
int count = foo.Count();
You should do this instead:
IQueryable<Foo> foo = (...);
Iqueryable<Foo> topOne = foo.OrderBy(x => x.Foo).Take(1);
int count = foo.Count();

I am afraid there is no easy way to remove the OrderBy operator from queryable.
What you can do, however, is to re-create the IQueryable based on the new expression obtained from rewriting queryable.Expression(see here) omitting the OrderBy call.

If you can't eliminate the root cause, here is a workaround:
totalRecordCount = queryable.OrderBy(x => 0).Count();
SQL Server's query optimizer will remove this useless ordering. It won't have runtime cost.

I think you have implemented you paging code wrongly. You actually need to query the database twice, once for the paged datasource and once for the total row count. This is how the setup should look.
public IList<MyObj> GetPagedData(string filter, string sort, int skip, int take)
{
using(var db = new DataContext())
{
var q = GetDataInternal(db);
if(!String.IsNullOrEmpty(filter))
q = q.Where(filter); //Using Dynamic linq
if(!String.IsNullOrEmpty(sort))
q = q.OrderBy(sort); //And here
return q.Skip(skip).Take(take).ToList();
}
}
public int GetTotalCount(string filter)
{
using(var db = new DataContext())
{
var q = GetDataInternal(db);
if(!String.IsNullOrEmpty(filter))
q = q.Where(filter); //Using Dynamic linq
return q.Count(); //Without ordering and paging.
}
}
private static IQuerable<MyObj> GetDataInternal(DataContext db)
{
return
from x in db.JournalEventsView
where ...
select new ...;
}
The filtering and sorting is done using the Dynamic linq library

I know it is not quite what you are looking for, but index on [DataOwnerID] with inclusion of DataTimeStamp could make your query less expensive.

How to perform .Max() on a property of all objects in a collection and return the object with maximum value [duplicate]

This question already has answers here:
How to use LINQ to select object with minimum or maximum property value
(20 answers)
Closed 7 years ago.
I have a list of objects that have two int properties. The list is the output of another linq query. The object:
public class DimensionPair
{
public int Height { get; set; }
public int Width { get; set; }
}
I want to find and return the object in the list which has the largest Height property value.
I can manage to get the highest value of the Height value but not the object itself.
Can I do this with Linq? How?

We have an extension method to do exactly this in MoreLINQ. You can look at the implementation there, but basically it's a case of iterating through the data, remembering the maximum element we've seen so far and the maximum value it produced under the projection.
In your case you'd do something like:
var item = items.MaxBy(x => x.Height);
This is better (IMO) than any of the solutions presented here other than Mehrdad's second solution (which is basically the same as MaxBy):
It's O(n) unlike the previous accepted answer which finds the maximum value on every iteration (making it O(n^2))
The ordering solution is O(n log n)
Taking the Max value and then finding the first element with that value is O(n), but iterates over the sequence twice. Where possible, you should use LINQ in a single-pass fashion.
It's a lot simpler to read and understand than the aggregate version, and only evaluates the projection once per element

This would require a sort (O(n log n)) but is very simple and flexible. Another advantage is being able to use it with LINQ to SQL:
var maxObject = list.OrderByDescending(item => item.Height).First();
Note that this has the advantage of enumerating the list sequence just once. While it might not matter if list is a List<T> that doesn't change in the meantime, it could matter for arbitrary IEnumerable<T> objects. Nothing guarantees that the sequence doesn't change in different enumerations so methods that are doing it multiple times can be dangerous (and inefficient, depending on the nature of the sequence). However, it's still a less than ideal solution for large sequences. I suggest writing your own MaxObject extension manually if you have a large set of items to be able to do it in one pass without sorting and other stuff whatsoever (O(n)):
static class EnumerableExtensions {
public static T MaxObject<T,U>(this IEnumerable<T> source, Func<T,U> selector)
where U : IComparable<U> {
if (source == null) throw new ArgumentNullException("source");
bool first = true;
T maxObj = default(T);
U maxKey = default(U);
foreach (var item in source) {
if (first) {
maxObj = item;
maxKey = selector(maxObj);
first = false;
} else {
U currentKey = selector(item);
if (currentKey.CompareTo(maxKey) > 0) {
maxKey = currentKey;
maxObj = item;
}
}
}
if (first) throw new InvalidOperationException("Sequence is empty.");
return maxObj;
}
}
and use it with:
var maxObject = list.MaxObject(item => item.Height);

Doing an ordering and then selecting the first item is wasting a lot of time ordering the items after the first one. You don't care about the order of those.
Instead you can use the aggregate function to select the best item based on what you're looking for.
var maxHeight = dimensions
.Aggregate((agg, next) =>
next.Height > agg.Height ? next : agg);
var maxHeightAndWidth = dimensions
.Aggregate((agg, next) =>
next.Height >= agg.Height && next.Width >= agg.Width ? next: agg);

And why don't you try with this ??? :
var itemsMax = items.Where(x => x.Height == items.Max(y => y.Height));
OR more optimise :
var itemMaxHeight = items.Max(y => y.Height);
var itemsMax = items.Where(x => x.Height == itemMaxHeight);
mmm ?

The answers so far are great! But I see a need for a solution with the following constraints:
Plain, concise LINQ;
O(n) complexity;
Do not evaluate the property more than once per element.
Here it is:
public static T MaxBy<T, R>(this IEnumerable<T> en, Func<T, R> evaluate) where R : IComparable<R> {
return en.Select(t => new Tuple<T, R>(t, evaluate(t)))
.Aggregate((max, next) => next.Item2.CompareTo(max.Item2) > 0 ? next : max).Item1;
}
public static T MinBy<T, R>(this IEnumerable<T> en, Func<T, R> evaluate) where R : IComparable<R> {
return en.Select(t => new Tuple<T, R>(t, evaluate(t)))
.Aggregate((max, next) => next.Item2.CompareTo(max.Item2) < 0 ? next : max).Item1;
}
Usage:
IEnumerable<Tuple<string, int>> list = new[] {
new Tuple<string, int>("other", 2),
new Tuple<string, int>("max", 4),
new Tuple<string, int>("min", 1),
new Tuple<string, int>("other", 3),
};
Tuple<string, int> min = list.MinBy(x => x.Item2); // "min", 1
Tuple<string, int> max = list.MaxBy(x => x.Item2); // "max", 4

I believe that sorting by the column you want to get the MAX of and then grabbing the first should work. However, if there are multiple objects with the same MAX value, only one will be grabbed:
private void Test()
{
test v1 = new test();
v1.Id = 12;
test v2 = new test();
v2.Id = 12;
test v3 = new test();
v3.Id = 12;
List<test> arr = new List<test>();
arr.Add(v1);
arr.Add(v2);
arr.Add(v3);
test max = arr.OrderByDescending(t => t.Id).First();
}
class test
{
public int Id { get; set; }
}

In NHibernate (with NHibernate.Linq) you could do it as follows:
return session.Query<T>()
.Single(a => a.Filter == filter &&
a.Id == session.Query<T>()
.Where(a2 => a2.Filter == filter)
.Max(a2 => a2.Id));
Which will generate SQL like follows:
select *
from TableName foo
where foo.Filter = 'Filter On String'
and foo.Id = (select cast(max(bar.RowVersion) as INT)
from TableName bar
where bar.Name = 'Filter On String')
Which seems pretty efficient to me.

Based on Cameron's initial answer, here is what I've just added at my enhanced version of SilverFlow library's FloatingWindowHost (copying from FloatingWindowHost.cs at http://clipflair.codeplex.com source code)
public int MaxZIndex
{
get {
return FloatingWindows.Aggregate(-1, (maxZIndex, window) => {
int w = Canvas.GetZIndex(window);
return (w > maxZIndex) ? w : maxZIndex;
});
}
}
private void SetTopmost(UIElement element)
{
if (element == null)
throw new ArgumentNullException("element");
Canvas.SetZIndex(element, MaxZIndex + 1);
}
Worth noting regarding the code above that Canvas.ZIndex is an attached property available for UIElements in various containers, not just used when being hosted in a Canvas (see Controlling rendering order (ZOrder) in Silverlight without using the Canvas control). Guess one could even make a SetTopmost and SetBottomMost static extension method for UIElement easily by adapting this code.

You can also upgrade Mehrdad Afshari's solution by rewriting the extention method to faster (and better looking) one:
static class EnumerableExtensions
{
public static T MaxElement<T, R>(this IEnumerable<T> container, Func<T, R> valuingFoo) where R : IComparable
{
var enumerator = container.GetEnumerator();
if (!enumerator.MoveNext())
throw new ArgumentException("Container is empty!");
var maxElem = enumerator.Current;
var maxVal = valuingFoo(maxElem);
while (enumerator.MoveNext())
{
var currVal = valuingFoo(enumerator.Current);
if (currVal.CompareTo(maxVal) > 0)
{
maxVal = currVal;
maxElem = enumerator.Current;
}
}
return maxElem;
}
}
And then just use it:
var maxObject = list.MaxElement(item => item.Height);
That name will be clear to people using C++ (because there is std::max_element in there).