If I have a LINQ to Entities query that returns some results, does using Take improve the performance? I mean because at the end of the query we are saying .Take(thisManyRecords) then does it first return all the records and then just returns back to me thisManyRecords or it really from beginning limits its search to thisManyRecords?
Linq to entities implies EF and some provider that gets queries.
You dont need bags of data to check. SQL profiler will show the SQL statement used.
If you apply .Take to an IQueryable source. You end up with an Expression tree and this can be sent to db ;
Context.Set().Where(t => ??).OrderBy(t => t.??).Take(n);
results in TOP(n) being sent.
IF you however force the enumeration eg .ToList() and the did .Take() then this applies to the memory objects and is not sent to DB.
Query Execution in EF explained
Related
I am working with MVC Core with EF-6.x and I am using Lambda expression, but my senior told me use LINQ why, because while using lambda with where clause it will pull all the data from database and later it will apply where condition. In case of LINQ if you use where condition it will pull only filtered data. Please let me know what is correct?
e.gLambda: context.tablename.where(condition);// Should I go with this
LINQ: (from T in tablename where t.(condition));// or this?
Please let me know what is correct?
e.gLambda: context.tablename.where(condition);// Should I go with this
LINQ: (from T in tablename where t.(condition));// or this?
Short answer: it doesn't really matter. Since context.tablename ultimately returns an IQueryable<T>, Entityframework will not try to hit the database until you try to iterate the final result from your expression, not to mention, .ToArray() and .ToList() each, does that iteration for you.
Either you used LINQ expression syntax (which gets compiled as LINQ methods) or LINQ methods, when you attempt to begin iterating the results, Entityframework creates an Expression tree for you underneath the hood that consists of your query altogether (including Wheres, Joins, GroupBys, etc). Since the structure of a LINQ might not percisely match the structure of a SQL query (or whatever data-source query), depending on the source (i.e database, e.g SQL Server), Entityframework then attempts to make the best possible optimization to your expression tree so that its structure would match an executable query on the source (e.g SQL Server query). Finally, it translates that expression tree into an actual query and executes it against the data source, and return your data (after doing all the mapping of course).
If you really, and I mean REALLY want to go through the complications of how an IQueryable engine works, I'd suggest taking a walk through Matt Warren's series: 'LINQ: Building an IQueryable provider series'. That's not actually Entityframework but it shares the same concept.
Both syntax will be translated into the same IL code. The difference, if the filter will be applied on server or client side is, if the source is IQueryable<T> or IEnumerable<T>.
my senior told me use LINQ why, because while using lambda with where
clause it will pull all the data from database and later it will apply
where condition.
Your senior is wrong, you can use the statement you prefer they are the same, both will filter the data at database level. If you have Sql Server as database server, you can use Sql Server Profiler to dump the queries executed by both statements and you will see that they are the same.
I have a question about LINQ pagination and stored procedure pagination:
If I am using LINQ pagination, then what logic use Microsoft? Means it load all data into list and then performed pagination logic?
For example, if I have 1 million records in a SQL Server database and I want to use LINQ pagination then how will it work?
Code snippet:
_list = _list.Skip(StartIndex).Take(FetchRecords).ToList();
If LINQ loads all records first, then do I have to go for stored procedure pagination?
You need to remember that
The ToList(IEnumerable) method forces immediate query evaluation and returns a List that contains the query results.
https://learn.microsoft.com/en-us/dotnet/api/system.linq.enumerable.tolist
ie without ToList method call, query is not executed hence no data.
Secondly you should also use OrderBy method eg. OrderBy(o => o.ID); as you should explicitly know in which order items are skipped and taken.
Further consider using 'where' and 'select' method to limit the data payload.
Are this 2 queries functionally equivalent?
1)
var z=Categories
.Where(s=>s.CategoryName.Contains("a"))
.OrderBy(s => s.CategoryName).AsEnumerable()
.Select((x,i)=>new {x.CategoryName,Rank=i});
2)
var z=Categories.AsEnumerable()
.Where(s=>s.CategoryName.Contains("a"))
.OrderBy(s => s.CategoryName)
.Select((x,i)=>new {x.CategoryName,Rank=i});
I mean, does the order of "AsNumerable()" in the query change the number of data items retrieved from the client, or the way they are retrieved?
Thank you for you help.
Are this 2 queries functionally equivalent?
If by equivalent you means to the final results, then probably yes (depending how the provider implements those operations), the difference is in the second query you are using in-memory extensions.
I mean, does the order of "AsNumerable()" in the query change the
number of data items retrieved from the client, or the way they are
retrieved?
Yes, in the first query, Where and OrderBy will be translated to SQL and the Select will be executed in memory.
In your second query all the information from the database is brought to memory, then is filtered and transformed in memory.
Categories is probably an IQueryable, so you will be using the extensions in Queryable class. this version of the extensions receive a Expression as parameter, and these expression trees is what allows transform your code to sql queries.
AsEnumerable() returns the object as an IEnumerable, so you will be using the extensions in Enumerable class that are executed directly in memory.
Yes they do the same thing but in different ways. The first query do all the selection,ordering and conditions in the SQL database itself.
However the second code segment fetches all the rows from the database and store it in the memory. Then after it sorts, orders, and apply conditions to the fetched data i.e now in the memory.
AsEnumerable() breaks the query into two parts:
The Inside-Part(query before AsEnumerable) is executed as LINQ-to-SQL
The Outside-Part(query after AsEnumerable) is executed as LINQ-to-Objects
I am fetching a list of products including their prices. I want to get just enable prices.
I wrote two type of queries:
context.Products.Include("Prices").Where(p=>p.Prices.Where(pr=>pr.Enable==true).Count()>0).ToList();
And the other one is:
context.Products.Include("Prices").ToList().RemoveAll(p => p.Prices.Where(pr => pr.Enable == true).ToList().Count == 0);
Which one is more optimized?
Assuming you are using an EntityFramework context, the first one is way better.
This is because Linq to SQL will translate the statement into an SQL statement. The Where statements will result in an according SQL Where. So only the necessary subset of the elements are retrieved.
The second statement retrieves all Products and Prices and then removes the unwanted elements.
This assumes that you have a remote database. If your database is running locally or you already have all Products and Prices in memory its not so easy to tell (you would have to use the profiler for that).
This kind of question really depends on a lot of things, so it is not so easy to say which is better.
But from the code, the first one is doing the where clause at sql side, where the second code is getting all the data out from sql and do the where in application.
so it will depend on the sql server, the application hardware and data amount.
I have many queries to do and I was wondering if there is a significant performance difference between querying a List and a DataTable or even a SQL server indexed table? Or maybe would it be faster if I go with another type of collection?
In general, what do you think?
Thank you!
It should almost always be faster querying anything in memory, like a List<T> or a DataTable vis-a-vis a database.
Having said that, you have to get the data into an in-memory object like a List before it can be queried, so I certainly hope you're not thinking of dumping your DB into a List<T> for fast querying. That would be a very bad idea.
Am I getting the point of your question?
You might be confusing Linq with a database query language. I would suggest reading up on Linq, particularly IQueryable vs IEnumerable.
In short, Linq is an in-code query language, which can be pointed at nearly any collection of data to perform searches, projections, aggregates, etc in a similar fashion as SQL, but not limited to RDBMSes. It is not, on its face, a DB query language like SQL; it can merely be translated into one by use of an IQueryable provider, line Linq2SQL, Linq2Azure, Linq for Entities... the list goes on.
The IEnumerable side of Linq, which works on in-memory objects that are already in the heap, will almost certainly perform better than the IQueryable side, which exists to be translated into a native query language like SQL. However, that's not because of any inherent weakness or strength in either side of the language. It is instead a factor of (usually) having to send the translated IQueryable command over a network channel and get the results over same, which will perform much more slowly than your local computer's memory.
However, the "heavy lifting" of pulling records out of a data store and creating in-memory object representations has to be done at some time, and IQueryable Linq will almost certainly be faster than instantiating ALL records as in-memory objects, THEN using IEnumerable Linq (Linq 2 Objects) to filter to get your actual data.
To illustrate: You have a table MyTable; it contains a relatively modest 200 million rows. Using a Linq provider like Linq2SQL, your code might look like this:
//GetContext<>() is a method that will return the IQueryable provider
//used to produce MyTable entitiy objects
//pull all records for the past 5 days
var results = from t in Repository.GetContext<MyTable>()
where t.SomeDate >= DateTime.Today.AddDays(-5)
&& t.SomeDate <= DateTime.Now
select t;
This will be digested by the Linq2SQL IQueryable provider into a SQL string like this:
SELECT [each of MyTable's fields] FROM MyTable WHERE SomeDate Between #p1 and #p2; #p1 = '2/26/2011', #p2 = '3/3/2011 9:30:00'
This query can be easily digested by the SQL engine to return EXACTLY the information needed (say 500 rows).
Without a Linq provider, but wanting to use Linq, you may do something like this:
//GetAllMyTable() is a method that will execute and return the results of
//"Select * from MyTable"
//pull all records for the past 5 days
var results = from t in Repository.GetAllMyTable()
where t.SomeDate >= DateTime.Today.AddDays(-5)
&& t.SomeDate <= DateTime.Now
select t;
On the surface, the difference is subtle. Behind the scenes, the devil's in those details. This second query relies on a method that retrieves and instantiates an object for every record in the database. That means it has to pull all those records, and create a space in memory for them. That will give you a list of 200 MILLION records, which isn't so modest anymore now that each of those records was transmitted over the network and is now taking up residence in your page file. The first query MAY introduce some overhead in building and then digesting the expression tree into SQL, but it's MUCH preferred over dumping an entire table into an in-memory collection and iterating over it.