Develop Reference

force to use join in nhibernate syntax instead of iteration

i am trying to figure out why nhibernate query iterates on values instead of using joins internally. due to this iteration it becomes slower, cause it will iterates all values one by one.
i.e. it generates n no. of queries and execute it one by one instead of joins.
documentClrType is evaluated dynamically at runtime, so i can't use directly in QueryOver<> syntax
documentClrType is FactSheetPrivate as of now.
i observed query in logger it is created something like this.
select * from foo where col1=#val1
select * from foo where col1=#val2
select * from foo where col1=#val3
select * from foo where col1=#val4
so,
how could i turn this query to joins instead of iterations?
what could be syntax for this dynamic types with query over?
i am bit new to nhibernate, any guidance will be appreciated.
var criteria=
store.Session.
CreateCriteria(documentClrType)
.Add(Restrictions.Disjunction()
.Add(Restrictions.Le("CreationDate",)DateTime.Now )
.Add(Restrictions.Le("AccurateDate",)DateTime.Now )
)
criteria=criteria.CreateCriteria("Entity")
.Add(Restrictions.Eq("DBTypeString",receiverType))
return criteria.List<IDocument>()
// at this lines instead of join query iterates value one by one
following screen display entity nhibernate xml file
following screen display packets nhibernate XML file

NHibernate has a feature to batch the selects
configuration.SetProperty(NHibernate.Cfg.Environment.DefaultBatchFetchSize, "20")
or use eager fetching
return criteria.SetFetchMode("TheReferenceOrCollectionProperty", FetchMode.Eager).List<IDocument>()

Entity Framework Core count does not have optimal performance

I need to get the amount of records with a certain filter.
Theoretically this instruction:
_dbContext.People.Count (w => w.Type == 1);
It should generate SQL like:
Select count (*)
from People
Where Type = 1
However, the generated SQL is:
Select Id, Name, Type, DateCreated, DateLastUpdate, Address
from People
Where Type = 1
The query being generated takes much longer to run in a database with many records.
I need to generate the first query.
If I just do this:
_dbContext.People.Count ();
Entity Framework generates the following query:
Select count (*)
from People
.. which runs very fast.
How to generate this second query passing search criteria to the count?

There is not much to answer here. If your ORM tool does not produce the expected SQL query from a simple LINQ query, there is no way you can let it do that by rewriting the query (and you shouldn't be doing that at the first place).
EF Core has a concept of mixed client/database evaluation in LINQ queries which allows them to release EF Core versions with incomplete/very inefficient query processing like in your case.
Excerpt from Features not in EF Core (note the word not) and Roadmap:
Improved translation to enable more queries to successfully execute, with more logic being evaluated in the database (rather than in-memory).
Shortly, they are planning to improve the query processing, but we don't know when will that happen and what level of degree (remember the mixed mode allows them to consider query "working").
So what are the options?
First, stay away from EF Core until it becomes really useful. Go back to EF6, it's has no such issues.
If you can't use EF6, then stay updated with the latest EF Core version.
For instance, in both v1.0.1 and v1.1.0 you query generates the intended SQL (tested), so you can simply upgrade and the concrete issue will be gone.
But note that along with improvements the new releases introduce bugs/regressions (as you can see here EFCore returning too many columns for a simple LEFT OUTER join for instance), so do that on your own risk (and consider the first option again, i.e. Which One Is Right for You :)

Try to use this lambda expression for execute query faster.
_dbContext.People.select(x=> x.id).Count();

Try this
(from x in _dbContext.People where x.Type == 1 select x).Count();
or you could do the async version of it like:
await (from x in _dbContext.People where x.Type == 1 select x).CountAsync();
and if those don't work out for you, then you could at least make the query more efficient by doing:
(from x in _dbContext.People where x.Type == 1 select x.Id).Count();
or
await (from x in _dbContext.People where x.Type == 1 select x.Id).CountAsync();

If you want to optimize performance and the current EF provider is not not (yet) capable of producing the desired query, you can always rely on raw SQL.
Obviously, this is a trade-off as you are using EF to avoid writing SQL directly, but using raw SQL can be useful if the query you want to perform can't be expressed using LINQ, or if using a LINQ query is resulting in inefficient SQL being sent to the database.
A sample raw SQL query would look like this:
var results = _context.People.FromSql("SELECT Id, Name, Type, " +
"FROM People " +
"WHERE Type = #p0",
1);
As far as I know, raw SQL queries passed to the FromSql extension method currently require that you return a model type, i.e. returning a scalar result may not yet be supported.
You can however always go back to plain ADO.NET queries:
using (var connection = _context.Database.GetDbConnection())
{
connection.Open();
using (var command = connection.CreateCommand())
{
command.CommandText = "SELECT COUNT(*) FROM People WHERE Type = 1";
var result = command.ExecuteScalar().ToString();
}
}

It seems that there has been some problem with one of the early releases of Entity Framework Core. Unfortunately you have not specified exact version so I am not able to dig into EF source code to tell what exactly has gone wrong.
To test this scenario, I have installed the latest EF Core package and managed to get correct result.
Here is my test program:
And here is SQL what gets generated captured by SQL Server Profiler:
As you can see it matches all the expectations.
Here is the excerpt from packages.config file:
...
<package id="Microsoft.EntityFrameworkCore" version="1.1.0" targetFramework="net452" />
...
So, in your situation the only solution is to update to the latest package which is 1.1.0 at the time of writing this.

Does this get what you want:
_dbContext.People.Where(w => w.Type == 1).Count();

I am using EFCore 1.1 here.
This can occur if EFCore cannot translate the entire Where clause to SQL. This can be something as simple as DateTime.Now that might not even think about.
The following statement results in a SQL query that will surprisingly run a SELECT * and then C# .Count() once it has loaded the entire table!
int sentCount = ctx.ScheduledEmail.Where(x => x.template == template &&
x.SendConfirmedDate > DateTime.Now.AddDays(-7)).Count();
But this query will run an SQL SELECT COUNT(*) as you would expect / hope for:
DateTime earliestDate = DateTime.Now.AddDays(-7);
int sentCount = ctx.ScheduledEmail.Where(x => x.template == template
&& x.SendConfirmedDate > earliestDate).Count();
Crazy but true. Fortunately this also works:
DateTime now = DateTime.Now;
int sentCount = ctx.ScheduledEmail.Where(x => x.template == template &&
x.SendConfirmedDate > now.AddDays(-7)).Count();

sorry for the bump, but...
probably the reason the query with the where clause is slow is because you didnt provide your database a fast way to execute it.
in case of the select count(*) from People query we dont need to know the actual data for each field and we can just use a small index that doesnt have all these fields in them so we havent got to spend our slow I/O on. The database software would be clever enough to see that the primary key index requires the least I/O to do the count on. The pk id's require less space than the full row so you get more back to count per I/O block so you can complete faster.
Now in the case of the query with the Type it needs to read the Type to determine it's value. You should create an index on Type if you want your query to be fast or else it will have to do a very slow full table scan, reading all rows. It helps when your values are more discriminating. A column Gender (usually) only has two values and isnt very discriminating, a primary key column where every value is unique is highly dscriminating. Higher discriminating values will result in a shorter index range scan and a faster result to the count.

What I used to count rows using a search query was
_dbContext.People.Where(w => w.Type == 1).Count();
This can also be achieved by
List<People> people = new List<People>();
people = _dbContext.People.Where(w => w.Type == 1);
int count = people.Count();
This way you will get the people list too if you need it further.

Nhibernate Linq Sum with calculation causes NotSupportedException

I am using Nhibernate 3.3.3.4001 with SybaseSQLAnywhere12Dialect and trying to use a simple calculation within a sum linq function.
I would like to use Linq rather than HQL to achieve this and would be happy to extend Nhibernate with a default linq to hql generator.
Using the following Linq query raises the exception
System.NotSupportedException: Expression type 'NhSumExpression' is not supported by this SelectClauseVisitor
Linq Query
var query = (from c in session.Query<Entity>()
select c.FieldValue / 100M).Sum()
Expected SQL Statement
SELECT SUM(FieldValue / 100.0) FROM TableName
If the same query is run without the Sum() function it performs the calculation correctly, as such am confused why the Sum() wouldn't work.

You may try to rewrite your LINQ query a bit, just like this one:
var result = ((from c in session.Query<Entity>()
select c.FieldValue).Sum()) / 100;
Resulting query could also be optimized by moving division operator out of the sum:
SELECT SUM(FieldValue) / 100 FROM TableName

There is an open issue when using constants within Linq group function (Max, Min, Sum) functions. If another field is used it will work correctly.
Found bug within NHibernate 3.3.3
https://nhibernate.jira.com/browse/NH-3376

LINQ to SQL under the hood

I know this was partly asked before, but none of the questions completely answer this.
What happens when one uses LINQ to SQL to retrieve data from the database?
I've read the following questions:
Optimizing a LINQ to SQL query
Linq-To-Sql optimization for queries
What is unclear to me is: at which point is the database accessed? When are the queries run?
If I run the following query, how will it translate to a SQL query?
DatabaseDataContext db = new DatabaseDataContext();
var users = from x in db.Users
where x.Rank > 10
orderby x.RegistrationDate descending
select x)
.Skip(pageIndex * recordCount)
.Take(recordCount);
And then, later, if I try to access some property of some user, how will the query be constructed (this is partly answered here)?
var temp = users.ToList()[0].SomeProperty;
Basically, what I really want to know is how the LINQ to SQL works under the hood, how it goes from the C# language statement to the SQL and how it is optimized.

The LINQ to SQL framework will take your Linq query, which is in fact an expression tree, and then will convert this expression tree into a pure SQL query. See How to: Use Expression Trees to Build Dynamic Queries
In fact, every expression tree can be translated into whatever langage or database you need. You will have different providers implementing IQueryable for different databases (Oracle, SQLite, etc.). Note that LINQ to SQL is the abbreviation of LINQ to SQL Server. On the other hand, Entity framework/LINQ to Entities can be extent more easily for other databases.
The main point here is the IQueryable interface, which contains an expression tree, and also the implementation of the provider. For an example on how to implement a provider, ie how to translate from an expression tree to a query, see LINQ: Building an IQueryable Provider
Here is a snippet that will give you a flavor of what happens under the hood:
if (select.OrderBy != null && select.OrderBy.Count > 0)
{
this.AppendNewLine(Indentation.Same);
sb.Append("ORDER BY ");
for (int i = 0, n = select.OrderBy.Count; i < n; i++)
{
OrderExpression exp = select.OrderBy[i];
if (i > 0)
{
sb.Append(", ");
}
this.Visit(exp.Expression);
if (exp.OrderType != OrderType.Ascending)
{
sb.Append(" DESC");
}
}
}

The queries are run as soon as you demand the result.
var qry = (from x in db.Users where x.Rank > 10 orderby x.RegistrationDate descending
select x)
at this point the query has not run, becuase you haven't used the result.
Put it in a foreach or transfor it to a List and the query is forced to Materiliaze.
The rule of thumb is:
Whenever GetEnumerator is called on an IQueryable - the query is forced to materiliaze (wich meens "to to the database and get the actual recourds")

All you want to know is answered in the article on MSDN about LINQ to SQL: http://msdn.microsoft.com/en-us/library/bb425822.aspx
By the way, if you're only going to use a part of your result, as in your code above, it's better to modify your query, like so:
var prop = (from x in db.Users
where x.Rank > 10
orderby x.RegistrationDate descending
select x.SomeProperty)
.Skip(pageIndex)
.First()
.Select(x => x);
Optimization you do in your query is often more important than how the system performs peephole optimization under the hood...

Advanced Linq sorting C#

I have an IQueryable that has a list of pages.
I want to do: Pages.OrderByDescending(o => CalculateSort(o.page));
the method calculate sort is similar to that here is a plain english version:
public int calculatesort(page p)
{
int rating = (from r in db.rating select r). sum();
int comments = //query database for comments;
float timedecayfactor = math.exp(-page.totalhoursago);
return sortscore = (rating +comments)* timedecayfactor;
}
when I run a code similar to the one above an error is thrown that the mothode calculatesort cannot be converted to sql.
How can I do a conver the function above to be understood by sql so that I can use it to sort the pages?
Is this not a good approach for large data? Is there another method used to sort sets of results other than dynamically at the database?
I havent slept for days trying to fix this one :(

your code is nowhere near compiling so I'm guessing a lot here but I hope this gives an idea none the less.
As several have posted you need to give Linq-2-Sql an expression tree. Using query syntax that's what happens (by compiler magic)
from p in pages
let rating = (from r in db.rating
where r.PageId == p.PageId
select r.Value).Sum()
let comments = (from c in db.Comments
where c.PageId == p.PageId
select 1).Count()
let timedecayfactor = Math.Exp(-(p.totalhoursago))
orderby (rating + comments)*timedecayfactor descending
select p;
I haven't actually tried this against a database, there's simply too many unknown based on your code, so there might still be stuff that can't be translated.

The error occurs because LINQ cannot convert custom code/methods into SQL. It can convert only Expression<Func<>> objects into SQL.
In your case, you have a complex logic to do while sorting, so it might make sense to do it using a Stored Procedure, if you want to do it in the DB Layer.
Or load all the objects into main memory, and run the calculate sort method on the objects in memory
EDIT :
I don't have the code, so Describing in english is the best I can do :
Have table with structure capable of temporarily storing all the current users data.
Have a calculated field in the Pages table that holds the value calculated from all the non-user specific fields
Write a stored procedure that uses values from these two sources (temp table and calc field) to actually do the sort.
Delete the temp table as the last part in the stored proc
You can read about stored procs here and here

var comments = db.comments.Where(...);
Pages.OrderByDescending(p=>(db.rating.Sum(r=>r.rate) + comments.Count()) * Math.Exp(-p.totalhoursago))

Linq is expecting Calculatesort to return a "queryable" expression in order to generate its own SQL.
In can embed your 'calculatesort' method in this lambda expression. (I replaced your variables with constants in order to compile in my environment)
public static void ComplexSort(IQueryable<string> Pages)
{
Pages.OrderByDescending(p =>
{
int rating = 99;//(from r in db.rating select r). sum();
int comments = 33;//query database for comments;
double timedecayfactor = Math.Exp(88);
return (rating + comments) * timedecayfactor;
});
}
Also, you can even try to run that in parallel (since .net 4.0) replacing the first line with
Pages.AsParallel().OrderByDescending(p =>

Yes, counting previous answers: the LINQ to SQL doesn't know how to translate CalculateSort method. You should convert LINQ to SQL to ordinary LINQ to Object before using custom method.
Try to use this in the way you call the CalculateSort by adding AsEnumerable:
Pages.AsEnumerable().OrderByDescending(o => CalculateSort(o.page));
Then you're fine to use the OrderByDescending extension method.
UPDATE:
LINQ to SQL will always translate the query in the code into Expression tree. It's quite almost the same concept as AST of any programming language. These expression trees are further translated into SQL expression specific to SQL Server's SQL, because currently LINQ to SQL only supports SQL Server 2005 and 2008.