How to get Total row Count & the records from the below query?
Contracts cont = db.contracts.SqlQuery("SELECT TOP (20) *
FROM (SELECT
ROW_NUMBER() OVER (ORDER BY dbo.Contracts.expDate desc) As RowID,
dbo.Contracts.*,
TotalRows=COUNT(*) OVER()
FROM dbo.Contracts
Where dbo.Contracts.cancelled = 1) as temp
WHERE temp.RowID >" + 20).ToList();
I'm getting the records but don't know how to get the Total row Count.
Can Any body suggest best method to get the Total row Count & the records from the above query?
Your code won't work because you're returning a list of Contracts AND a count, but you're trying to assign it to only a Contracts. You need to project to an anonymous type, or create a custom type to project to that includes both the count and a collection of Contracts.
Why do you insist on using a sql query? This should do the same thing.
var contracts = (from x in db.contacts where x.cancelled == 1
orderby x.expDate descending
select new { Count=x.Count(), Records=x.Skip(20).Take(20) }).ToList();
Unless you want the total rows without the where clause, in which case it would be:
var contracts = (from x in db.contacts orderby x.expDate descending
select new { Count=x.Count(),
Records=x.Where(y => y.canceled == 1).Skip(20).Take(20) }).ToList();
Related
Thanks in advance for taking time to read this question.
I have a view in my database, lets call it Members_VW
In my .net 5 API, I'm trying to get a paginated response for the list of members from the view with search parameters. I need to also return the total number of responses for the front end to know in how many pages the results will be returned in.
Currently the Members_VW is made with a query like:
select
col1, col2, col3
from
table1 1
inner join table2 2 on 1.key = 2.key
inner join tble3 3 on 3.key = 2.key
where
defaultcondition1 = '1'
and
defaultcondition2 = '2'
I referred to this answer and tried using CTE which ended up changing my view to using a query like this:
with cte1 as (
select
col1, col2, col3
from
table1 1
inner join table2 2 on 1.key = 2.key
inner join tble3 3 on 3.key = 2.key
where
defaultcondition1 = '1'
and
defaultcondition2 = '2')
cte2 as (
select count(*) over() from cte1 )
select
*
from
cte1, cte2
But this didn't work because it would always return the total number of rows in cte1 without any of the filters applied.
So, I continued to try to construct queries to return the total number of rows after the conditions are applied and found that this query works:
select
col1, col2, col3, count(*) over()
from
table1 1
inner join table2 2 on 1.key = 2.key
inner join tble3 3 on 3.key = 2.key
where
defaultcondition1 = '1'
and
defaultcondition2 = '2'
Currently, I'm trying to implement the same query with EF Core but am struggling to implement that.
I've tried implementing the solution provided here, but as one of the comments suggests, this implementation is no longer allowed.
I am trying to avoid an implementation where I use a raw query. Is there anyway to get the result from count(*) over() without using a raw query?
The following is my current implementation:
IQueryable<MembersVW> membersQuery = _context.MembersVW;
membersQuery = membersQuery.Where(u => u.MemberId == memberid);
membersQuery = membersQuery.OrderBy(m => m.MemberId).Skip(page * size).Take(size);
When I do:
membersQuery = membersQuery.Count()
I'm returned with the following error:
Error CS0029 Cannot implicitly convert type 'int' to 'System.Linq.IQueryable<PersonalPolicyAPI.Models.VwPersonalPolicyMember>'
Again, thanks for reading my question, appreciate any help you can offer. 🙏🏾
I've read your question about can it be done with one query. While I'm not aware of any way to do it with 1 query I can offer one more solution that will help with your concern about performance and 2 queries. I do this frequently. 😁 Try:
//execute both queries at the same time instead of sequentially
var countqry = membersQuery.CountAsync();
var pageqry = membersQuery.OrderBy(m => m.MemberId).Skip(page * size).Take(size).ToListAsync();
//wait for them both to complete
Task.WaitAll(countqry, pageqry);
//use the results
var count = countqry.Result;
var page = pageqry.Result;
membersQuery.Count() returns integer not the queryable
you can do
int count = membersQuery.Count();
List<MemberVW> = membersQuery.OrderBy(m => m.MemberId).Skip(page * size).Take(size).ToList();
and you can return with
public class MemberVwWithCount {
public int Count{get;set;}
public List<MemberVW> Members {get; set;}
}
You try to assign the Count Value, which is an Integer, to the variable of your query, which is an IQueryable. That's all there is to it.
If you want to do it in one single query, as you suggest in one of your comments, you can first execute the query to get all Entries, then count the result, and then filter the result with skip/take. This is most probably not the most efficient way to do this, but it should work.
I'd also suggest to use AsNoTracking() if you do not modify any data in this function/api.
EDIT:
I'd suggest this solution for now. The counting is fast, as it actually doesn't fetch any data and just counts the rows. It is still two queries tho, gonna try to combine it & edit my answer later.
var count = await yourContext.YourTable.CountAsync();
var data = await yourContext.YourTable
.OrderBy(x => x.YourProp)
.Skip(10).Take(10)
//.AsNoTracking()
.ToListAsync();
EDIT2:
Okay, so, I couldn't get it to just make on DB-Call yet, however, I could combine it syntactically. However, the approach in my first edit is easier to read and does basically the same. Still, gonna dig deeper into this, there's gotta be a funky way to do this.
var query = yourContext.YourTable.AsQueryable();
var result = await query.OrderBy(x => x.Prop)
.Select(x => new {Data = x, Count = query.Count()} )
.Skip(50).Take(50)
.AsNoTracking()
.ToListAsync();
var count = result.FirstOrDefault()?.Count ?? 0; //If empty/null return 0
var data = result.Select(x => x.Data).ToList();
In membersQuery = membersQuery.Count() line you are assigning integer value to a queryable list, which is incorrect. You can get the list item counts after your query like this i.e.
membersQuery = membersQuery.OrderBy(m => m.MemberId).Skip(page * size).Take(size);
int totalCount = membersQuery.Count();
To get count column in same list, you first need to add Count property in your MembersVW class and then use LINQ projection to add column value.
Solution-1:
memberQuery = membersQuery.Select(p => new MembersVW
{
col1 = p.col1
col2 = p.col2
col3 = p.col3
count = totalCount
});
Solution-2:
With LINQ foreach loop i.e.
membersQuery.ForEach(item =>
{
item.count = totalCount;
});
I have some Ids store in below variable:
List<int> Ids;
Now I want to get records based on above Ids but with same order as it is in above Ids.
For eg: Records are like this in database:
Employee:
Id
1
2
3
4
5
Now if Ids array holds Ids like this : 4,2,5,3,1 then I am trying to get records in this order order only:
Query:
var data = context.Employee.Where(t => Ids.Contains(t.Id)).ToList();
But above query is giving me output like it is in table:
Id
1
2
3
4
5
Expected output :
Id
4
2
5
3
1
Update:I have already tried this below solution but as this is entity framework it didn't work out:
var data = context.Employee.Where(t => Ids.Contains(t.Id))
.OrderBy(d => Ids.IndexOf(d.Id)).ToList();
For above solution to make it working I have to add to list :
var data = context.Employee.Where(t => Ids.Contains(t.Id)).ToList()
.OrderBy(d => Ids.IndexOf(d.Id)).ToList();
But I don't want to load data in memory and then filter out my record.
Since the order in which the data is returned when you do not specify an ORDER BY is not determined, you have to add an ORDER BY to indicate how you want it sorted. Unfortunately you have to order based on objects/values in-memory, and cannot use that to order in your SQL query.
Therefore, the best you can do is to order in-memory once the data is retrieved from the database.
var data = context.Employee
// Add a criteria that we only want the known ids
.Where(t => Ids.Contains(t.Id))
// Anything after this is done in-memory instead of by the database
.AsEnumerable()
// Sort the results, in-memory
.OrderBy(d => Ids.IndexOf(d.Id))
// Materialize into a list
.ToList();
Without stored procedures you can use Union and ?: that are both canonical functions.
I can't immagine other ways.
?:
You can use it to assign a weigth to each id value then order by the weigth. Also, you have to generate ?: using dynamic linq.
What is the equivalent of "CASE WHEN THEN" (T-SQL) with Entity Framework?
Dynamically generate LINQ queries
Union
I think this is the more simple way to obtain it. In this case you can add a Where/Union for each Id.
EDIT 1
About using Union you can use code similar to this
IQueryable<Foo> query = context.Foos.AsQueryable();
List<int> Ids = new List<int>();
Ids.AddRange(new[] {3,2,1});
bool first = true;
foreach (int id in Ids)
{
if (first)
{
query = query.Where(_ => _.FooId == id);
first = false;
}
else
{
query = query.Union(context.Foos.Where(_ => _.FooId == id));
}
}
var results = query.ToList();
This generate the followiong query
SELECT
[Distinct2].[C1] AS [C1]
FROM ( SELECT DISTINCT
[UnionAll2].[C1] AS [C1]
FROM (SELECT
[Distinct1].[C1] AS [C1]
FROM ( SELECT DISTINCT
[UnionAll1].[FooId] AS [C1]
FROM (SELECT
[Extent1].[FooId] AS [FooId]
FROM [Foos] AS [Extent1]
WHERE [Extent1].[FooId] = #p__linq__0
UNION ALL
SELECT
[Extent2].[FooId] AS [FooId]
FROM [Foos] AS [Extent2]
WHERE [Extent2].[FooId] = #p__linq__1) AS [UnionAll1]
) AS [Distinct1]
UNION ALL
SELECT
[Extent3].[FooId] AS [FooId]
FROM [Foos] AS [Extent3]
WHERE [Extent3].[FooId] = #p__linq__2) AS [UnionAll2]
) AS [Distinct2]
p__linq__0 = 3
p__linq__1 = 2
p__linq__2 = 1
EDIT 2
I think the best approach is in memory approach because it has the same network load, EF does not generate the ugly query that could not work on databases different from SQL Server and code is more readable. In your particular application could be that union/where is better. So, generally I would suggest you to try memory approach then, if you have [performance] issues, you can check if union/where is better.
I want to get top five most repeating records from a table in link to Entity Framework 4.0. How it can be possible in a single query which returns a list of collection of five records?
You simply group by count, order descending by count and then Take(5). Grouping examples, amongst others, can be found at 101 LINQ Samples.
Actually you should group by fields which define whether record is repeating or not. E.g. in your case it should be something like member id. Then you can introduce new range variable which will keep number of records in each group. Use that variable for ordering results:
var query = from s in db.Statistics
group s by s.MemberId into g // group by member Id
let loginsCount = g.Count() // get count of entries for each member
orderby loginsCount descending // order by entries count
select new { // create new anonymous object with all data you need
MemberId = g.Key,
LoginsCount = loginsCount
};
Then take first 5:
var top5 = query.Take(5);
That will generate query like
SELECT TOP (5) // Take(5)
[GroupBy1].[K1] AS [MemberId], // new { MemberId, LoginsCount }
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
[Extent1].[MemberId] AS [K1],
COUNT(1) AS [A1] // let loginsCount = g.Count()
FROM [dbo].[Statistics] AS [Extent1]
GROUP BY [Extent1].[MemberId] // group s by s.MemberId
) AS [GroupBy1]
ORDER BY [GroupBy1].[A1] DESC // orderby loginsCount descending
I'm using Entity Framework to get the total row count for a table. I simply want the row count, no where clause or anything like that. The following query works, but is slow. It took about 7 seconds to return the count of 4475.
My guess here is that it's iterating through the entire table, just like how IEnumerable.Count() extension method works.
Is there a way I can get the total row count "quickly"? is there a better way?
public int GetLogCount()
{
using (var context = new my_db_entities(connection_string))
{
return context.Logs.Count();
}
}
You can even fire Raw SQL query using entity framework as below:
var sql = "SELECT COUNT(*) FROM dbo.Logs";
var total = context.Database.SqlQuery<int>(sql).Single();
That is the way to get your row count using Entity Framework. You will probably see faster performance on the second+ queries as there is an initialization cost the first time that you run it. (And it should be generating a Select Count() query here, not iterating through each row).
If you are interested in a faster way to get the raw row count in a table, then you might want to try using a mini ORM like Dapper or OrmLite.
You should also make sure that your table is properly defined (at the very least, that it has a Primary Key), as failure to do this can also affect the time to count rows in the table.
If you have access to do so, it would be much quicker to query the sys tables to pull this information.
E.g.
public Int64 GetLogCount()
{
var tableNameParam = new SqlParameter("TableName", "Logs");
var schemaNameParam = new SqlParameter("SchemaName", "dbo");
using (var context = new my_db_entities(connection_string))
{
var query = #"
SELECT ISNULL([RowCount],0)
FROM (
SELECT SchemaName,
TableName,
Sum(I.rowcnt) [RowCount]
FROM sysindexes I
JOIN sysobjects O (nolock) ON I.id = o.id AND o.type = 'U'
JOIN (
SELECT so.object_id,
ss.name as SchemaName,
so.name as TableName
FROM sys.objects SO (nolock)
JOIN sys.schemas SS (nolock) ON ss.schema_id = so.schema_id
) SN
ON SN.object_id = o.id
WHERE I.indid IN ( 0, 1 )
AND TableName = #TableName AND SchemaName = #SchemaName
GROUP BY
SchemaName, TableName
) A
";
return context.ExecuteStoreQuery<Int64>(query, tableNameParam, schemaNameParam).First();
}
}
I'm trying to determine how to count the matching rows on a table using the EntityFramework.
The problem is that each row might have many megabytes of data (in a Binary field). Of course the SQL would be something like this:
SELECT COUNT(*) FROM [MyTable] WHERE [fkID] = '1';
I could load all of the rows and then find the Count with:
var owner = context.MyContainer.Where(t => t.ID == '1');
owner.MyTable.Load();
var count = owner.MyTable.Count();
But that is grossly inefficient. Is there a simpler way?
EDIT: Thanks, all. I've moved the DB from a private attached so I can run profiling; this helps but causes confusions I didn't expect.
And my real data is a bit deeper, I'll use Trucks carrying Pallets of Cases of Items -- and I don't want the Truck to leave unless there is at least one Item in it.
My attempts are shown below. The part I don't get is that CASE_2 never access the DB server (MSSQL).
var truck = context.Truck.FirstOrDefault(t => (t.ID == truckID));
if (truck == null)
return "Invalid Truck ID: " + truckID;
var dlist = from t in ve.Truck
where t.ID == truckID
select t.Driver;
if (dlist.Count() == 0)
return "No Driver for this Truck";
var plist = from t in ve.Truck where t.ID == truckID
from r in t.Pallet select r;
if (plist.Count() == 0)
return "No Pallets are in this Truck";
#if CASE_1
/// This works fine (using 'plist'):
var list1 = from r in plist
from c in r.Case
from i in c.Item
select i;
if (list1.Count() == 0)
return "No Items are in the Truck";
#endif
#if CASE_2
/// This never executes any SQL on the server.
var list2 = from r in truck.Pallet
from c in r.Case
from i in c.Item
select i;
bool ok = (list.Count() > 0);
if (!ok)
return "No Items are in the Truck";
#endif
#if CASE_3
/// Forced loading also works, as stated in the OP...
bool ok = false;
foreach (var pallet in truck.Pallet) {
pallet.Case.Load();
foreach (var kase in pallet.Case) {
kase.Item.Load();
var item = kase.Item.FirstOrDefault();
if (item != null) {
ok = true;
break;
}
}
if (ok) break;
}
if (!ok)
return "No Items are in the Truck";
#endif
And the SQL resulting from CASE_1 is piped through sp_executesql, but:
SELECT [Project1].[C1] AS [C1]
FROM ( SELECT cast(1 as bit) AS X ) AS [SingleRowTable1]
LEFT OUTER JOIN (SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
COUNT(cast(1 as bit)) AS [A1]
FROM [dbo].[PalletTruckMap] AS [Extent1]
INNER JOIN [dbo].[PalletCaseMap] AS [Extent2] ON [Extent1].[PalletID] = [Extent2].[PalletID]
INNER JOIN [dbo].[Item] AS [Extent3] ON [Extent2].[CaseID] = [Extent3].[CaseID]
WHERE [Extent1].[TruckID] = '....'
) AS [GroupBy1] ) AS [Project1] ON 1 = 1
[I don't really have Trucks, Drivers, Pallets, Cases or Items; as you can see from the SQL the Truck-Pallet and Pallet-Case relationships are many-to-many -- although I don't think that matters. My real objects are intangibles and harder to describe, so I changed the names.]
Query syntax:
var count = (from o in context.MyContainer
where o.ID == '1'
from t in o.MyTable
select t).Count();
Method syntax:
var count = context.MyContainer
.Where(o => o.ID == '1')
.SelectMany(o => o.MyTable)
.Count()
Both generate the same SQL query.
I think you want something like
var count = context.MyTable.Count(t => t.MyContainer.ID == '1');
(edited to reflect comments)
As I understand it, the selected answer still loads all of the related tests. According to this msdn blog, there is a better way.
http://blogs.msdn.com/b/adonet/archive/2011/01/31/using-dbcontext-in-ef-feature-ctp5-part-6-loading-related-entities.aspx
Specifically
using (var context = new UnicornsContext())
var princess = context.Princesses.Find(1);
// Count how many unicorns the princess owns
var unicornHaul = context.Entry(princess)
.Collection(p => p.Unicorns)
.Query()
.Count();
}
This is my code:
IQueryable<AuctionRecord> records = db.AuctionRecord;
var count = records.Count();
Make sure the variable is defined as IQueryable then when you use Count() method, EF will execute something like
select count(*) from ...
Otherwise, if the records is defined as IEnumerable, the sql generated will query the entire table and count rows returned.
Well, even the SELECT COUNT(*) FROM Table will be fairly inefficient, especially on large tables, since SQL Server really can't do anything but do a full table scan (clustered index scan).
Sometimes, it's good enough to know an approximate number of rows from the database, and in such a case, a statement like this might suffice:
SELECT
SUM(used_page_count) * 8 AS SizeKB,
SUM(row_count) AS [RowCount],
OBJECT_NAME(OBJECT_ID) AS TableName
FROM
sys.dm_db_partition_stats
WHERE
OBJECT_ID = OBJECT_ID('YourTableNameHere')
AND (index_id = 0 OR index_id = 1)
GROUP BY
OBJECT_ID
This will inspect the dynamic management view and extract the number of rows and the table size from it, given a specific table. It does so by summing up the entries for the heap (index_id = 0) or the clustered index (index_id = 1).
It's quick, it's easy to use, but it's not guaranteed to be 100% accurate or up to date. But in many cases, this is "good enough" (and put much less burden on the server).
Maybe that would work for you, too? Of course, to use it in EF, you'd have to wrap this up in a stored proc or use a straight "Execute SQL query" call.
Marc
Use the ExecuteStoreQuery method of the entity context. This avoids downloading the entire result set and deserializing into objects to do a simple row count.
int count;
using (var db = new MyDatabase()){
string sql = "SELECT COUNT(*) FROM MyTable where FkId = {0}";
object[] myParams = {1};
var cntQuery = db.ExecuteStoreQuery<int>(sql, myParams);
count = cntQuery.First<int>();
}
I think this should work...
var query = from m in context.MyTable
where m.MyContainerId == '1' // or what ever the foreign key name is...
select m;
var count = query.Count();