After a little bit of advise.
We are producing a standard list of data with filters.
This filtered list can have 3 alternative views which represent the data in different manor
my question is:
should the stored procedure return the data in the format that is required for each view for the front end?
or should the business layer re-format the data?
I have always believed it best to get in and out of the database as fast as possible, and allow the business layer to handle formatting.
Thanks in advance for any advise
UPDATE:-
by formating,
1 view contain all data withing a scrollable table
2 view will group data withing headings, (click on a heading to expand to view)
3 view will group data into date groups and display dates as table row headers (7 day intivials) and the data count will be displayed within those intivals
I have always believed it best to get in and out of the database as
fast as possible, and allow the business layer to handle formatting.
Short answer: Correct, you have answered your own question.
The database returns data to your application, how it is presented is up to the application. What happens when you want to supply the same data to another application? Do you want to have to write procedures every time you want the data in a different format? Of course not.
1: view contain all data withing a scrollable table
You cannot return a scrolling window from a procedure. Anyway, how would the database know how much data will need to be scrolled? It does not know the size of window/viewport you are using.
2: view will group data withing headings, (click on a heading to expand to view)
This requires user interaction which is solely the domain of the presentation layer.
3: view will group data into date groups and display dates as table
row headers (7 day intivials) and the data count will be displayed
within those intivals
You can return multiple result sets from a procedure, to give the results and the counts, but again this is more easily handled in the application.
In general you should use the presentation layer to format your data output. You can have the stored procedure return the raw data set and then set up, for example 3 different XSLT files for the 3 different presentation types. This way you can always do DataTable.ToXML and apply the XSLT template to this output to produce the format that the user needs.
Related
Here's a problem I experience (simplified example):
Let's say I have several tables:
One customer can have mamy products and a product can have multiple features.
On my asp.net front-end I have a grid with customer info:
something like this:
Name Address
John 222 1st st
Mark 111 2nd st
What I need is an ability to filter customers by feature. So, I have a dropdown list of available features that are connected to a customer.
What I currently do:
1. I return DataTable of Customers from stored procedure. I store it in viewstate
2. I return DataTable of features connected to customers from stored procedure. I store it in viewstate
3. On filter selected, I run stored procedure again with new feature_id filter where I do joins again to only show customers that have selected feature.
My problem: It is very slow.
I think that possible solutions would be:
1. On page load return ALL data in one viewstate variable. So basically three lists of nested objects. This will make my page load slow.
2. Perform async loazing in some smart way. How?
Any better solutions?
Edit:
this is a simplified example, so I also need to filter customer by property that is connected through 6 tables to table Customer.
The way I deal with these scenarios is by passing in Xml to SQL and then running a join against that. So Xml would look something like:
<Features><Feat Id="2" /><Feat Id="5" /><feat Id="8" /></Features>
Then you can pass that Xml into SQL (depending on what version of SQL there are different ways), but in the newer version's its a lot easier than it used to be:
http://www.codeproject.com/Articles/20847/Passing-Arrays-in-SQL-Parameters-using-XML-Data-Ty
Also, don't put any of that in ViewState; there's really no reason for that.
Storing an entire list of customers in ViewState is going to be hideously slow; storing all information for all customers in ViewState is going to be worse, unless your entire customer base is very very small, like about 30 records.
For a start, why are you loading all the customers into ViewState? If you have any significant number of customers, load the data a page at a time. That will at least reduce the amount of data flowing over the wire and might speed up your stored procedure as well.
In your position, I would focus on optimizing the data retrieval first (including minimizing the amount you return), and then worry about faster ways to store and display it. If you're up against unusual constraints that prevent this (very slow database; no profiling tools; not allowed to change stored procedures) than please let us know.
Solution 1: Include whatever criteria you need to filter on in your query, only return and render the requested records. No need to use viewstate.
Solution 2: Retrieve some reasonable page limit of customers, filter on the browser with javascript. Allow easy navigation to the next page.
Hello i am developing a Database web application and i am having many reports to populate. I just want to know which one is the Best method among the following which will give me fast and accurate result as the data is going to be in 1000's.
Through populating Dataset?
Through DataReader ?
Through Array List ?
I am using 3 tier architecture. so what if i am writing a function which would be the most appropriate return type of the function in DATA Access Layer ?
You can use "push" method to set the data with a DataSet - this will give you the advantage to set the datasource for the main report and all subreports in one call to the database. However there are some limitations, for example you will be not able to use subreports in the details section.
I am not sure you can use datareader and array list as datasources. Even if you can I cannot see any advantages. Using datareader means that you will keep your connection to the database open while report is rendered ( the first pass). This may take some time and is not necessary. Array list ( if can be used) will allow you to set the data for one table - it is a flat structure - no relations. In most of the cases you probably will load the array-list from the database anyway so it will not make sense to get the data load it in an array and use the array to set one table if you can use a dataset.
Why you are ignoring the regular "pull" method ? It will be simpler.
Goal:
Have the latest data to be in the first row and not in the end of listview. The program is in WPF.
Problem:
Based on this source code, don't know how to recieve the latest data in the first row in the listview
For the sake of simplicity, that sample code is loading hard-coded data into the ListView. It isn't retrieving live data from a database, as you will presumably be doing.
If at all possible, you should sort the data when you retrieve it from the database. Then you'll have no problem just binding that sorted data to your ListView
Get the current date/time from your data source and sort on that, descending. For example, if you are reading the records from an SQL Server database, include a field GETDATE() AS date_time. If the data is from a file, use the built-in .NET date/time objects (Now in VB.NET) to add the timestamp.
You need to provide the data to the listview in a sorted fashion.
This example shows one way.
You could also expose the data through a viewmodel that would wrap some model (possibly whatever source list you're looking at), sort it (probably using a linq .orderyby method) and give the data to your view.
I have been tasked to write a module for importing data into a client's system.
I thought to break the process into 4 parts:
1. Connect to the data source (SQL, Excel, Access, CSV, ActiveDirectory, Sharepoint and Oracle) - DONE
2. Get the available tables/data groups from the source - DONE
i. Get the available fields form the selected table/data group - DONE
ii. Get all data from the selected fields - DONE
3. Transform data to the user's requirements
4. Write the transformed data the the MSSQL target
I am trying to plan how to handle complex data transformations like:
Get column A from Table tblA, inner joined to column FA from table tblB, and concatenate these two with a semicolon in between.
OR
Get column C from table tblC on source where column tblC.D is not in table tblG column G on target database.
My worry is not the visual, but the representation in code of this operation.
I am NOT asking for sample code, but rather for some creative ideas.
The data transformation will not be with free text, but drag and drop objects that represent actions.
I am a bit lost, and need some fresh input.
maybe you can grab some ideas from this open source project: Rhino ETL.
See my answer: Manipulate values in a datatable?
Hey everybody, got an interesting question I think. I've got a Silverlight3 application which is querying a SQL Server 2005 database asynchronously. However, some of the data sets the queries return are massive and so I'm looking into data pagination.
Goals:
1) Just in time data calls - I only want to query for page 3's data when the user clicks to go to page 3.
2) I want a slider to control which page I'm on - SliderControl is SL3 with its movement binded to a stored procedure call (is my initial guess as to an approach).
3) Read ahead data as a label for the slider. So the slider will say "Page 1 of 50" or perhaps "Gant - Hart". Some sort of indication as to where you are in the data without actually querying for all the data until the user drops the slider into position.
I'm almost positive that I need to do some sort of a query to get the total number of rows and some sort of bookmark data that the query will eventually return. Otherwise I don't know how to split up the slider nor would I be able to do page labels (The gant to hart stuff).
Anyone have experience with this kind of thing? Thanks.
There are usually at least 2 separate lookups involved here, the first to get the pager information and a second to retrieve the data appropriate to the current page.
The pager (slider) needs to know the total number results that would be returned if you weren't doing paging. This is needed to show the pretty "Total # of Results" indicator for your use and to calculate how many total pages you've got. In your case you're going to want to format a nice visual display of the pages for selection (determined by your desired records/page value) in your slider. That's your first query; it should return just a single scalar int or long and you can calculate what you need for your slider display.
The second results include the actual records to be displayed for a given page.
Step 1 of 2 for a page of data is to run a query to filter and order your results by a primary key present within the data you're returning, throwing the key values in a temp table with an AUTO_INCREMENT/IDENTITY field or getting the row number over for the results of a derived table. (Summary: 2 fields, 1 for sequence/1 for primary key to join in step 2).
Step 2 of 2 is to join the key values with your tables which contain data, ordering by the sequence determined in Step 1, selecting only rows numbered (desired start row) to (desired end row) as determined by the page number and page size you've selected.
A stored procedure is strongly advised to keep the data on the server during this operation while you're having to examine more data than you really care to bring back to optimize bandwidth.