I am trying to get the count of the items as I'm applying a query to a IQueryable.
I'am trying to do it like:
this.lblSth.Text = new Repository<Sth>().GetAll().Where(p => p.PersonId == personId).ToList().Count().ToString();
I think this gets all the data across the condition and takes the objects, then it takes the count; so I'm curious if for example I'd just take the Id columns and cast it to the list or some other smart way; that count operation would be quicker?
Info: GetAll() => It's a repository pattern method that returns IQueryable objects T from linqToSql data entity.
I'm open to all types of different ideas. Thanks
I think the call to Where and ToList is redundant.
see below.
this.lblSth.Text = new Repository<Sth>().GetAll().Count(p => p.PersonId == personId).ToString();
If you want to do this quicker, just don't call ToList():
this.lblSth.Text = new Repository<Sth>().GetAll()
.Where(p => p.PersonId == personId)
.Count()
.ToString();
This way, (assuming it's an SQL-backed IQueryable<T>) it will execute a query like SELECT COUNT(*) FROM …, not SELECT * FROM … like your approach. And this query should be much faster.
ToList() will execute the query and turn your IQUeryable into IEnumerable. I would call the count on the where clause. That way the Count will become part of the end query
Related
I have a feeling I know that what the reason is for this behavior, but I don't know what the best way of resolving it will be.
I have built a LinqToSQL query:
public IEnumerable<AllConditionByCountry> GenerateConditions(int paramCountryId)
{
var AllConditionsByCountry =
(from cd in db.tblConditionDescriptions...
join...
join...
select new AllConditionByCountry
{
CountryID = cd.CountryID,
ConditionDescription = cd.ConditionDescription,
ConditionID = cd.ConditionID,
...
...
}).OrderBy(x => x.CountryID).AsEnumerable<AllConditionByCountry>();
return AllConditionsByCountry;
}
This query returns about 9500+ rows of data.
I'm calling this from my Controller like so:
svcGenerateConditions generateConditions = new svcGenerateConditions(db);
IEnumerable<AllConditionByCountry> AllConditionsByCountry;
AllConditionsByCountry = generateConditions.GenerateConditions(1);
Which then I'm looping through:
foreach (var record in AllConditionsByCountry)
{
...
...
...
This is where I think the issue is:
var rList = AllConditionsByCountry
.Where(x => x.ConditionID == conditionID)
.Select(x => x)
.AsEnumerable();
I'm doing an nested loop based off the data that I'm gathering from the above query (utilizing the original data I'm getting from AllConditionByCountry. I think this is where my issue lies. When it is doing the filter on the data, it SLOWS down greatly.
Basically this process writes out a bunch of files (.json, .html)
I've tested this at first using just ADO.Net and to run through all of these records it took about 4 seconds. Using EF (stored procedure or LinqToSql) it takes minutes.
Is there anything I should do with my types of lists that I'm using or is that just the price of using LinqToSql?
I've tried to return List<AllConditionByCountry>, IQueryable, IEnumerable from my GenerateConditions method. List took a very long time (similar to what I'm seeing now). IQueryable I got errors when I tried to do the 2nd filter (Query results cannot be enumerated more than once).
I have run this same Linq statement in LinqPad and it returns in less than a second.
I'm happy to add any additional information.
Please let me know.
Edit:
foreach (var record in AllConditionsByCountry)
{
...
...
...
var rList = AllConditionsByCountry
.Where(x => x.ConditionID == conditionID)
.Select(x => x)
.AsEnumerable();
conditionDescriptionTypeID = item.ConditionDescriptionTypeId;
id = conditionDescriptionTypeID + "_" + count.ToString();
...
...
}
TL;DR: You're making 9895 queries against the database instead of one. You need to rewrite your query such that only one is executed. Look into how IEnumerable works for some hints into doing this.
Ah, yeah, that for loop is your problem.
foreach (var record in AllConditionsByCountry)
{
...
...
...
var rList = AllConditionsByCountry.Where(x => x.ConditionID == conditionID).Select(x => x).AsEnumerable();
conditionDescriptionTypeID = item.ConditionDescriptionTypeId;
id = conditionDescriptionTypeID + "_" + count.ToString();
...
...
}
Linq-to-SQL works similarly to Linq in that it (loosely speaking) appends functions to a chain to be executed when the enumerable is iterated - for example,
Enumerable.FromResult(1).Select(x => throw new Exception());
This doesn't actually cause your code to crash because the enumerable is never iterated. Linq-to-SQL operates on a similar principle. So, when you define this:
var AllConditionsByCountry =
(from cd in db.tblConditionDescriptions...
join...
join...
select new AllConditionByCountry
{
CountryID = cd.CountryID,
ConditionDescription = cd.ConditionDescription,
ConditionID = cd.ConditionID,
...
...
}).OrderBy(x => x.CountryID).AsEnumerable<AllConditionByCountry>();
You're not executing anything against a database, you're just instructing C# to build a query that does this when it is iterated. That's why just declaring this query is fast.
Your problem comes when you get to your for loop. When you hit your for loop, you signal that you want to start iterating the AllConditionsByCountry iterator. This causes .NET to go off and execute the initial query, which takes time.
When you call AllConditionsByCountry.Where(x => x.ConditionID == conditionID) in the for loop, you're constructing another iterator that doesn't actually do anything. Presumably you actually use the result of rList within that loop, however, you're essentially constructing N queries to be executed against the database (where N is the size of AllConditionsByCountry).
This leads to a scenario where you are effectively executing approximately 9501 queries against the database - 1 for your initial query and then one query for each element within the original query. The drastic slowdown compared to ADO.NET is because you're probably making 9500 more queries than you were originally.
You ideally should change the code so that there is one and only one query executed against the database. You've a couple of options:
Rewrite the Linq-to-SQL query such that all of the legwork is done by the SQL database
Rewrite the Linq-to-SQL query so it looks like this
var conditions = AllConditionsByCountry.ToList();
foreach (var record in conditions)
{
var rList = conditions.Where(....);
}
Note that in that example I am searching conditions rather than AllConditionsByCountry - .ToList() will return a list that has already been iterated so you do not create any more database queries. This will still be slow (since you're doing O(N^2) over 9500 records), but it will still be faster than creating 9500 queries since it will all be done in memory.
Just rewrite the query in ADO.NET if you're more comfortable with raw SQL than Linq-to-SQL. There's nothing wrong with this.
I think I should point out what methods cause an IEnumerable to be iterated and what ones don't.
Any method named As* (such as AsEnumerable<T>()) do not cause the enumerable to be iterated. It's essentially a way of casting from one type to another.
Any method named To* (such as ToList<T>()) will cause the enumerable to be iterated. In the event of Linq-to-SQL this will also execute the database query. Any method that also results in you getting a value out of the enumerable will also cause iteration. You can use this to your advantage by creating a query and forcing iteration using ToList() and then searching that list - this will cause the comparisons to be done in memory, which is what I demo above
//Firstly: IEnumerable<> should be List<>, because you need to massage result later
public IEnumerable<AllConditionByCountry> GenerateConditions(int paramCountryId)
{
var AllConditionsByCountry =
(from cd in db.tblConditionDescriptions...
join...
join...
select new AllConditionByCountry
{
CountryID = cd.CountryID,
ConditionDescription = cd.ConditionDescription,
ConditionID = cd.ConditionID,
...
...
})
.OrderBy(x => x.CountryID)
.ToList() //return a list, so only 1 query is executed
//.AsEnumerable<AllConditionByCountry>();//it's useless code, anyway.
return AllConditionsByCountry;
}
about this part:
foreach (var record in AllConditionsByCountry) // you can use AllConditionsByCountry.ForEach(record=>{...});
{
...
//AllConditionsByCountry will not query db again, because it's a list, no long a query
var rList = AllConditionsByCountry.Where(x => x.ConditionID == conditionID);//.Select(x => x).AsEnumerable(); //no necessary to use AsXXX if compilation do not require it.
...
}
BTW,
you should have your result paged, no page will need 100+ result. 10K return is the issue itself.
GenerateConditions(int paramCountryId, int page = 0, int pagesize = 50)
it's weird that you have to use a sub-query, usually it means GenerateConditions did not return the data structure you need, you should change it to give right data, no more subquery
use compiled query to improve more: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ef/language-reference/compiled-queries-linq-to-entities
we don't see your full query, but usually, it's right the part you should improve, especially when you have many conditions to filter and join and group... a little change could make all differences.
I got a product table and want to get datas sort by alphabetical. But when i write this query they are still coming by id. I check a lot of page in google but cant find any source.
var product = Db.tablename
.Where(s => s.colum == DropDownList2.SelectedValue)
.OrderBy(s=> s.Name);
This query
var product = Db.tablename
.Where(s => s.colum == DropDownList2.SelectedValue)
.OrderBy(s=> s.Name);
will not be executed until it is asked to. So you have to change it to the following one:
var product = Db.tablename
.Where(s => s.colum == DropDownList2.SelectedValue)
.OrderBy(s=> s.Name).ToList();
The reason why that happens is that actually you just have declared a query. I mean you haven't executed it. That's the nature of LINQ queries, which in technical terms is called deffered execution. On the other hand if you call the ToList() method at the end of your query, you will trigger the immediate execution of this query and it's result will be a List of the same type with s.Name.
You must use ToList to execute the sort.
var product = Db.tablename
.Where(s => s.colum == DropDownList2.SelectedValue)
.OrderBy(s=> s.Name).ToList();
The order by does nothing, just executes the query, the ToList will do the sort for the original query.
Assuming following tables
Person
id
name
PersonTeam
id
person_id
is_supervisor
team_id
Team
id
TimeSheet
id
team_id
I would like to obtain all TimeSheets for a supervisor. I got name of supervisor, then I need select which team he is got supervisor role. Then select all time sheet of those teams.
I believe following query does
var allTimeSheets = ctx.PersonTeam.Where(y => y.Person.name == supervisor_name).Where(x => x.is_supervisor == true).Select(z => z.Team).Select(t => t.TimeSheet);
afer this operation I cannot understand allTimeSheets is a
IQueryable<ICollection<TimeSheet>>
I expected more a
<ICollection<TimeSheet>>
or any IEnumrable.
Then questions are :
why I got that kind of result ?
how to obtain TimeSheet[] where I got IQueryable < ICollection < TimeSheet > > ?
why did I get that kind of result ? I expected more a ICollection<TimeSheet>
An IQueryable<T> is an IEnumerable<T>. The reason it's returning an IQueryable is so you can chain other methods like OrderBy onto it and project those to the actual SQL.
I just realized what you're asking. To "flatten" the collection of collections, use SelectMany instead of two chained Selects:
var allTimeSheets = ctx.PersonTeam
.Where(y => y.Person.name == supervisor_name
&& y.is_supervisor == true)
.SelectMany(z => z.Team, (z, t) => t.TimeSheet);
The answer to your second question still applies:
how do I obtain a TimeSheet[] from a IQueryable<ICollection<TimeSheet>>
(first of all use the first part to change to an IQueryable<TimeSheet>)
You can call one of the "conversion" methods like ToArray, ToList, to "hydrate" the query into a concrete type.
You can also call "AsEnumerableto cast to anIEnumerableto convert the query to Linq-To-Objects, which has better support for custom functions in sorts, filters, etc. Note that callingAsEnunerable` does no immediately fetch the objects, but will do as as soon as the collection in enumerated.
I am using LINQ to create a list. But I want to use a function at the end to generate the object iself, something LINQ complains about
LINQ to Entities does not recognize the method 'WashroomStatusItem GetWashroomStatusForItem(WashroomStatus)' method, and this method cannot be translated into a store expression.
What am I doing wrong?
var query = (from c in context.WashroomStatus
where c.WashroomId == GroupItem.WashroomID
select GetWashroomStatusForItem(c));
private WashroomStatusItem GetWashroomStatusForItem(WashroomStatus item)
{
WashroomStatusItem temp = new WashroomMonitorWCF.WashroomStatusItem();
//do stuff with it
return temp;
}
The problem is that the SQL conversion can't convert your method into SQL. You should use AsEnumerable() to "switch" from the out-of-process provider to LINQ to Objects. For example:
var query = context.WashroomStatus
.Where(c => c.WashroomId == GroupItem.WashroomID)
.AsEnumerable()
.Select(c => GetWashroomStatusForItem(c));
Note that if GetWashroomStatusForItem only uses some properties, you may want to project to those separately first, to reduce the amount of information fetched from the server:
var query = context.WashroomStatus
.Where(c => c.WashroomId == GroupItem.WashroomID)
.Select(c => new { c.Location, c.Date };
.AsEnumerable()
.Select(p => GetWashroomStatusForItem(p.Location, p.Date));
Jon Skeet's answer is correct, but I'd add that depending on the nature of GetWashroomStatusForItem(), it should probably either be broken down into LINQ statements and added into the query itself, or it should be executed after the query has returned.
So, lets say GetWashroomStatusForItem() looks something like this: note that this is extremely oversimplified.
public static WashroomStatus GetWashroomStatusForItem(Item c)
{
return c.WashroomStatus;
}
it should just be added to the LINQ query like this:
var query = (from c in context.WashroomStatus
where c.WashroomId == GroupItem.WashroomID
select c.WashroomStatus);
But if it relies heavily on stuff not in the db, I'd just end the Linq statement before you get the WashroomStatus, and then call GetWashroomStatusForItem() on the results. It's not gonna a performance difference since Linq uses lazy evaluation, and you generally want to keep db operations separate from "programmatic" ones.
Here is my query -
var data = Goaldata.GroupBy(c => c.GoalId).ToList();
This returns a Igrouping object and I want an Iqueryable object which I can directly query to get the data while in this case I have to loop through using a foreach() and then get the data. Is there another way to group by in LINQ which returns directly as a list of Iqueryable or a List as similar to what happens for order by in LINQ.
The easiest way is probably
var data = Goaldata.GroupBy(c => c.GoalId).SelectMany(c => c).ToList();
In the OO sense they aren't really grouped, but they are ordered with the groups together.
Whilst the accepted answer is correct, it seems to be unnecessarily complicated. Assuming GoalId is an int you can just use OrderBy:
var data = Goaldata.OrderBy(c => c.GoalId).ToList();
Or .GroupBy(c => c.GoalId).AsQueryable()...