I currently have a datagrid bound to a table with tens of thousands of records. I display the datagrid using the existing asp.net 2.0. I show ten records at a time.
My problem is whenever I try to access the next page, I gets all the records again from the database and then displays the ones that are required. This is slowing down the app. Is there feature within .net 2.0 which will help me optimize this issue? I cant use third party controls or ajax.
UPDATE
I cant use SQL to get 10 records at a time as well. Without changing any existing business logic or data retrieval, I want to do this.
You can do paging in your SQL and/or stored procedure.
Basically, you pass the sproc the number of items on the page and what page you are on. For example, page 3 and 20 records per page.
If you can use SQL Server 2005 or higher, you can use some newer keywords that make the query easier.
Here is a simple version of such a query:
SELECT ClientName, RowNumber
FROM (SELECT ClientName, ROW_NUMBER() OVER (ORDER BY ClientName) AS RowNumber
FROM Clients) AS cl
WHERE RowNumber BETWEEN 12 AND 30
You would make the values above of "12" and "30" be input parameters.
This way, you are only returning the rows you are going to display.
You can stick your datasource in the session.
then check if your DataSource is null from the session and then go get it.
This way you only get your data one time and if you need to get fresh data (say because you updated something) just empty the session variable that holds your data source.
EDIT:
here's a half attempt at an example, (by this i mean I'm not going into how to change the page because I assume you already know how to do that)
protected void PageChanging(object sender, GridViewPageEventArgs e)
{
if(Session["YourSessionVariableForData"] == null)
Session["YourSessionVariableForData"] = YourDataCall();
YourGridView.PageIndex = e.NewPageIndex;
YourGridView.DataSource = ((YourDataType)Session["YourSessionVariableForData"]);
YourGridView.DataBind();
}
Do the Paging in a Stored Procedure. Look into using the ROW_NUMBER() function for this. Create a class that will call the stored procedure and wire it to the grid using an ObjectDataSource.
Related
This past week I was tasked with moving a PHP based database to a new SQL database. There are a handful of requirements, but one of those was using ASP.Net MVC to connect to the SQL database...and I have never used ASP.Net or MVC.
I have successfully moved the database to SQL and have the foundation of the ASP site set up (after spending many hours pouring through tutorials). The issue I am having now is that one of the pages is meant to display a handful of fields (User_Name, Work_Date, Work_Description, Work_Location, etc) but the only way of grabbing all of those fields is by combining two of the tables. Furthermore, I am required to allow the user to search the combined table for any matching rows between a user inputted date range.
I have tried having a basic table set up that displays the correct fields and have implemented a search bar...but that only allows me to search by a single date, not a range. I have also tried to use GridView with its Query Builder feature to grab the data fields I needed (which worked really well), but I can't figure out how to attach textboxes/buttons to the newly made GridView. Using a single table with GridView works perfectly and using textboxes/buttons is very intuitive. I just can't seem to make the same connection with a joined view.
So I suppose my question is this: what is the best way for me to combine these two tables while also still having the ability to perform searches on the displayed data? If I could build this database from scratch I would have just made a table with the relevant data attached to it, but because this is derived from a previously made database it has 12+ years of information that I need to dump into it.
Any help would be greatly appreciated. I am kind of dead in the water here. My inexperience with these systems is getting the better of me. I could post the code that I have, but I am mainly interested in my options and then I can do the research on my own.
Thanks!
It's difficult to offer definitive answers to your questions due to the need for guesswork.
But here are some hints.
You can say WHERE datestamp >= '2017-01-01' AND datestamp < '2018-01-01' to filter all the rows in calendar year 2017. Many variations on this sort of date range filter are available.
Your first table probably has some kind of ID number on each row. Let's call it first.first_id. Your second table probably has its own id, let's call it second.second_id. And, it probably has another id that identifies a row in your first table, let's call it second.first_id. That second.first_id is called a foreign key in the second table to the first table. There can be any number of rows in your second table corresponding to your first table via this foreign key.
If this is the case you can do something like this:
SELECT first.datestamp, first.val1, first.val2, second.val1, second.val2
FROM first
JOIN second ON first.first_id = second.first_id
WHERE first.datestamp >= '2018-06-01' AND first.datestamp < '2018-07-01'
AND (first.val1 = 'some search term' OR second.val1 = 'some search term')
ORDER BY first.datestamp
This makes a virtual table by joining together your two physical tables (FROM...JOIN...).
Then it filters the rows you want from that virtual table (FROM ...).
Then it puts them in the order you want (ORDER BY...).
Finally, it chooses the columns from the virtual table you want in your result set (SELECT ...).
SQL database servers (MySQL, SQL Server, postgreSQL, Oracle and the rest) are very smart about doing this sort of thing efficiently.
I'm using jquery datatables to display a grid which uses webapi to retrieve it's data. The webapi uses linq to query a mssql database and it neatly uses filtering, sorting and skip/take to assemble it's query on a well-indexed table containing about a million records (and growing). A common scenario.
And it performs really well. The browser has to wait about 50 ms for the response (while paginating for example) to return.
However, after I took a look with a profiling tool I noticed about 25 ms to be used just selecting the total rowcount of the table. Which I want to know because I want the datatable to display something like: "displaying row 1 to 10 of 45.000 filtered out of 1.000.000" needing the total count.
I don't actually need to know the precise total count (it's just informative) every trip from the server so I perhaps could keep the value server side and refresh it every second in a different task without it interfering with the data retrieval of datatables. I would just return the 'close enough' value of the total row count.
Is there a solid mechanism for that? I've tried to put the total rowcount in a static used by multiple users during multiple callbacks and every time it was requested a async task was fired to refresh it.
That feels icky however, sharing the static and having a different thread update it doesn't feel all that stable to me. I've looked at SqlDependency to push the recordcount every time it changes from my data to my domain model but that doesn't seem to support SELECT COUNT(Id) FROM TABLE scenarios.
Any thoughts?
You could use one of the system tables if possible. You could ping this every minute and stick it in the cache. This article has two that it claims are sufficient options:
--The way the SQL management studio counts rows (look at table properties, storage
--, row count). Very fast, but still an approximate number of rows.
SELECT CAST(p.rows AS float)
FROM sys.tables AS tbl
INNER JOIN sys.indexes AS idx ON idx.object_id = tbl.object_id and idx.index_id < 2
INNER JOIN sys.partitions AS p ON p.object_id=CAST(tbl.object_id AS int)
AND p.index_id=idx.index_id
WHERE ((tbl.name=N'Transactions'
AND SCHEMA_NAME(tbl.schema_id)='dbo'))
or
--Quick (although not as fast as method 2) operation and equally important, reliable.
SELECT SUM (row_count)
FROM sys.dm_db_partition_stats
WHERE object_id=OBJECT_ID('Transactions')
AND (index_id=0 or index_id=1);
Have you considered taking the count when a query is performed and then echoing the value out to your clients via SignalR?
Basically, when the LINQ call returns get a .Count() and hand off the value to a background thread to let SignalR notify the clients of the update, at the same time you return the data to the requesting client.
SignalR will activate a javascript function in all of the client pages, where you can then take the passed in value and display it somewhere on the page.
http://www.asp.net/signalr
I have an operation (That I can't change) that starts threads that make calls to our Oracle database to see if a certain hotel(s) has availability on a certain date.
If a date/hotel combination has availability, that thread returns information about the date/hotel in the form of a DataTable that is merged into a Main DataTable of results. Yes, I know ... I inherited this.
So I am trying to re-write this operation. I still must query Oracle in threads to get the availability information, but I want to display the data as it is returned (in chunks of 5, 10? I'm flexible), instead of having the user sit in front of the screen for up to 4 minutes before a complete result is spat out into a GridView.
How do I do this directly from an .aspx page so I can make a web service call and populate a grid (JqGrid?) with the results?
If I haven't provided enough information or described what I am trying to achieve, please let me know and I will elaborate.
Oracle provides a field on each row called "rowid"
(http://www.adp-gmbh.ch/ora/concepts/rowid.html)
The first time you send the query, send in the int (x) to define what the highest rownumber you want is. Have the service return the total number of rows and the first x rows.
Then, the 2nd time you send the query, get the next x rows, rinse and repeat.
Basically, you need to send an ajax query for rows x through y each time until you have them all loaded.
I would recommend paging as well, since users typically don't want to see hundreds of results at a time.
I have around 100 rows in database ( which can increase in future ) for different labels used in page. These rows are required on all pages. I want to do it in a way that I get these rows once from db and put some where in cache or something similar to that. So that i don't have to query database on every page refresh. And how can this cache be used by all site visitors?
You can use Cache Variables something like this :
Cache["COMMON_DATA"] = lstData (This can be any collection like LIST<Country>)
You can also use SqlCacheDependency in asp.net that will auto reset your cache if more rows are added in the the database
else you can update the Cache again manually after inserting data in the table
Caching in ASP.net and Application State would be a couple of ideas for you.
This is what I am working with to get back to the web dev world
ASP.Net with VS2008
Subsonic as Data Access Layer
SqlServer DB
Home Project description:
I have a student registration system. I have a web page which should display the student records.
At present I have a gridview control which shows the records
The user logs on and gets to the view students page. The gridview shows the students in the system, where one of the columns is the registration status of open, pending, complete.
I want the user to be able to apply dynamic sorting or filters on the result returned so as to obtain a more refined result they wish to see. I envisioned allowing user to filter the results by applying where clause or like clause on the result returned, via a dataset interface by a subsonic method. I do not want to query the databse again to apply the filter
example: initial query
Select * from studentrecords where date = convert(varchar(32,getdate(),101)
The user then should be able to applly filter on the resultset returned so that they can do a last name like '%Souza%'
Is this even possible and is binding a datasource to a gridview control the best approach or should i create a custom collection inheriting from collectionbase and then binding that to the gridview control?
PS: Sorry about the typo's. My machine is under the influence of tea spill on my laptop
I use LINQ-to-SQL, not Subsonic, so YMMV, but my approach to filtering has been to supply an OnSelecting handler to the data source. In LINQ-to-SQL, I'm able to replace the result with a reference to a DataContext method that returns a the result of applying a table-valued function. You might want to investigate something similar with Subsonic.
As tvanfosson said, LINQ is very well suited to making composable queries; you can do this either with fully dunamic TSQL that the base library generates, or via a UDF that you mark with [FunctionAttribute(..., IsComposable=true)] in the data-context.
I'm not familiar with Subsonic, so I can't advise there; but one other thought: in your "date = " code, you might consider declaring a datetime variable and assigning it first... that way the optimiser can usually do a better job of optimising it (the query is simpler, and there is no question whether it is converting the datetime (per row) to a varchar, or the varchar to a datetime). The most efficient way of getting just the date portion of something is to cast/floor/cast:
SET #today = GETDATE()
SET #today = CAST(FLOOR(CAST(#today as float)) as datetime)
[update re comment]
Re composable - I mean that this allows you to build up a query such that only the final query is executed at the database. For example:
var query = from row in ctx.SomeComplexUdf(someArg)
where row.IsOpen && row.Value > 0
select row;
might go down the the server via the TSQL:
SELECT u1.*
FROM dbo.SomeComplexUdf(#p1) u1
WHERE u1.IsOpen = 1 -- might end up parameterized
AND u1.Value > 0 -- might end up parameterized
The point here being that only the suitable data is returned from the server, rather than lots of data being returned, then thrown away. LINQ-to-SQL can do all sorts of things via compsable queries, including paging, sorting etc. By minimising the amount of data you load from the database you can make significant improvements in performance.
The alternative without composability is that it simply does:
SELECT u1.*
FROM dbo.SomeComplexUdf(#p1) u1
And then throws away the other rows in your web app... obviously, if you are expecting 20 open records and 10000 closed records, this is a huge difference.
How about something like this?
Rather than assigning a data source / table to your grid control, instead attach a 'DataView' to it.
Here's sort of a pseudocode example:
DataTable myDataTable = GetDataTableFromSomewhere();
DataGridView dgv = new DataGridView();
DataView dv = new DataView(myDataTable);
//Then you can specify things like:
dv.Sort = "StudentStatus DESC";
dv.Filter = "StudentName LIKE '" + searchName + '";
dgv.DataSource = dv;
...and so on.