Hello i am developing a Database web application and i am having many reports to populate. I just want to know which one is the Best method among the following which will give me fast and accurate result as the data is going to be in 1000's.
Through populating Dataset?
Through DataReader ?
Through Array List ?
I am using 3 tier architecture. so what if i am writing a function which would be the most appropriate return type of the function in DATA Access Layer ?
You can use "push" method to set the data with a DataSet - this will give you the advantage to set the datasource for the main report and all subreports in one call to the database. However there are some limitations, for example you will be not able to use subreports in the details section.
I am not sure you can use datareader and array list as datasources. Even if you can I cannot see any advantages. Using datareader means that you will keep your connection to the database open while report is rendered ( the first pass). This may take some time and is not necessary. Array list ( if can be used) will allow you to set the data for one table - it is a flat structure - no relations. In most of the cases you probably will load the array-list from the database anyway so it will not make sense to get the data load it in an array and use the array to set one table if you can use a dataset.
Why you are ignoring the regular "pull" method ? It will be simpler.
Related
TL;DR In a SSRS 2008 report which uses a custom assembly to do some extra calculations can I pass an entire report dataset as a method parameter?
Full story
I have an SSRS report with 3 datasets, each returned from an SQL query.
(In case it makes a difference to my question they're currently shared datasets although I'm sure local would work too)
The largest and primary dataset is a list of tasks which may or may not have been completed. I have information in here such as the ID, status, create date/time, target resolution hours etc of each task.
This dataset is displayed in a tablix and is the focus of the report.
The remaining two datasets are not displayed and are for reference. One is a simple one column query which returns a list of holiday dates for the UK. The other is a small table which contains our exact business hours.
At the moment I'm able to loop through the rows in the tablix of tasks and pass multiple values from the current row to a method. This is useful if I want to do some calculations based on data found only in the current row. For example I could take the create date/time and the response target hours and the assembly would return a target date/time for the current task. Cool so far.
I want to do a more complicated version of this where I not only pass in the row data but the 2 other datasets to get my return value. This is because in reality the due date calculation is much more complex and must take into account changing business hours and holidays from the other 2 datasets.
Can I pass a dataset as a method parameter to an assembly? Something like:
=Code.MyClass.MyMethod(val1, val2, dataset1, dataset2);.
I've been unable to find much definitive information on this. Nearly all tutorials demonstrate what I'm already doing by processing single rows. I'm sure I had an MSDN article that hinted this was not possible but I've lost it (helpful I know). There's a post on the Microsoft forums where a moderator says it's not possible. The general lack of information and tutorials suggests it's not possible or I'm doing this in the wrong way.
Any suggestions?
(I have alternate solutions such as having the assembly fetch the other datasets or just writing something outside SSRS but I'm not pursuing those until I knnow whether it can be done this way).
An older topic on the MSDN forums Iterate through rows of dataset in report's custom code offers a more definitive answer as well as a potential solution to this problem.
Passing the DataSet as an object or collection is not a possibility because:
A dataset in Reporting Services is not the same type of object as an ADO.Net dataset. A report dataset is an internal object managed by the SSRS runtime (it's actually derived from a DataReader object) and not an XML structure containing datatables, etc. and cannot be passed into the report's custom code.
The only way to effectively loop through the rows of a report dataset is to call a custom function or referenced method in a report data region expression. Using this technique, it may be possible to pass all of the the row and field information into a code structure, array or collection.
The hint given in the above statement suggests passing row and field information into a code structure. A contributor to the linked MSDN topic, Migeul Catalao developed a workaround using such an approach.
A real-world scenario of it's usage with example code demonstrating Migeul Catalao's solution can be found here.
Granted, it is still more of a row-by-row approach, so I would strongly suggest moving outside of SSRS and pursue alternative solutions.
Although I've accepted the other answer due to it being clear and helpful I didn't use that solution in the end (I was too stupid to understand it) and went for something else that works.
Disclaimer: This is a horrible hack. It works absolutely great in my scenario so I though I'd share in case it was useful to somebody else. There are many pitfalls here which could most likely be worked around given time.
I ended up following the advice in the comment given by Steven White and looking into LookupSet. This function allows you to query a dataset to return matching rows and a single column of data.
It looks like this:
LookupSet(Fields!ComparisonField.Value, // The value to search for, e.g '001'.
Fields!MatchField.Value, // The column to match on in the target dataset.
Fields!MyColumn.Value, // The column that will be returned.
"MyDataSet") // The dataset to search.
This returns a string array representing the returned values.
So far so good, but I needed ALL columns and rows. This is where the dirty hack appears in the form of string concatenation:
LookupSet(0, // Dummy ID 0.
0, // Matches the dummy ID 0 so all rows are returned.
Fields!Column1.Value + "[^]" // I concatenate all of the values into
+ Fields!Column2.Value + "[^]" // one string with the separator [^]
+ Fields!.Column3.Value, // so I can split them later.
"MyDataSet") // The dataset to query
I can now pass this to my custom assembly:
=MyAssemblyNamespace.Class.Method(LookupSet(0,0,Fields!Column1.Value..., "MyDataSet"), other, parameters, here)
Now in my C# method I have a generic object which after some reflection is actually an array of strings.
Cast to something useful:
var stringList = ((IEnumerable)MyDataSetObject).Cast<string>().ToList();
Split it:
foreach (var item in stringList)
{
var columns = item.Split(new[] { "[^]" }, StringSplitOptions.None);
// columns is a string[] which holds each column value for the current row
// So columns[0] is the value for column 1 in this row
// In my case I pushed the values to a DataTable row each time and built a datatable
// which when finished represented my dataset in full with all rows and columns.
}
I hope this makes sense to anyone trying to achieve a similar result.
What is the best way to retrieve a "X" number of random records using Entity Framework (EF5 if it's relevant). The value of "X" will be set based on where this will be used.
Is there a method for doing this built into EF or is best to pull down a result set and then use a C# random number function to pull the records. Or is there a method that I'm not thinking of?
On the off chance that it's relevant I have a table that stores images that I use for different usages (there is a FK to an image type table). The images that I use in my carousel on the homepage is what I'm wanting to add some variety to...consequently how "random" it is doesn't matter to me much. I'm just trying to get away from the same six or so pictures always being displayed. (Also, I'm not really interested in debating/discussing storing images in a table vs local storage.)
The solution needs to be one using EF via a LINQ statement. If this isn't directly possibly I may end up doing something SIMILAR to what #cmd has recommended in the comments. This would most likely be a matter of retrieving a record count...testing the PK to make sure the resulting object wasn't null and building a LIST of the X number of object's PKs to pass to front end. The carousel lazy loads the images so I don't actually need the image when I'm building the list that will be used by the carousel.
Can you just add an ORDER BY RAND() clause to your query?
See this related question: MySQL: Alternatives to ORDER BY RAND()
I'm new to n-tier enterprise development. I just got quite a tutorial just reading threw the 'questions that may already have your answer' but didn't find what I was looking for. I'm doing a geneology site that starts off with the first guy that came over on the boat, you click on his name and the grid gets populated with all his children, then click on one of his kids that has kids and the grid gets populated with his kids and so forth. Each record has an ID and a ParentID. When you choose any given person, the ID is stored and then used in a search for all records that match the ParentID which returns all the kids. The data is never changed (at least by the user) so I want to just do one database access, fill all fields into one datatable and then do a requery of it each time to get the records to display. In the DAL I put all the records into a List which, in the ObjectDataSource the function that fills the GridView just returns the List of all entries. What I want to do is requery the datatable, fill the list back up with the new query and display in the GridView. My code is in 3 files here
(I can't get the backticks to show my code in this window) All I need is to figure out how to make a new query on the existing DataTable and copy it to a new DataTable. Hope this explains it well enough.
[edit: It would be easier to just do a new query from the database each time and it would be less resource intensive (in the future if the database gets too large) to store in memory, but I just want to know if I can do it this way - that is, working from 1 copy of the entire table] Any ideas...
Your data represents a tree structure by nature.
A grid to display it may not be my first choice...
Querying all data in one query can be done by using a complex SP.
But you are already considering performance. Thats always a good thing to keep in mind when coming up with a design. But creating something, improve it and only then start to optimize seems a better to go.
Since relational databases are not real good on hierarchical data, consider a nosql (graph)database. As you mentioned there are almost no writes to the DB, nosql shines here.
Here's a problem I experience (simplified example):
Let's say I have several tables:
One customer can have mamy products and a product can have multiple features.
On my asp.net front-end I have a grid with customer info:
something like this:
Name Address
John 222 1st st
Mark 111 2nd st
What I need is an ability to filter customers by feature. So, I have a dropdown list of available features that are connected to a customer.
What I currently do:
1. I return DataTable of Customers from stored procedure. I store it in viewstate
2. I return DataTable of features connected to customers from stored procedure. I store it in viewstate
3. On filter selected, I run stored procedure again with new feature_id filter where I do joins again to only show customers that have selected feature.
My problem: It is very slow.
I think that possible solutions would be:
1. On page load return ALL data in one viewstate variable. So basically three lists of nested objects. This will make my page load slow.
2. Perform async loazing in some smart way. How?
Any better solutions?
Edit:
this is a simplified example, so I also need to filter customer by property that is connected through 6 tables to table Customer.
The way I deal with these scenarios is by passing in Xml to SQL and then running a join against that. So Xml would look something like:
<Features><Feat Id="2" /><Feat Id="5" /><feat Id="8" /></Features>
Then you can pass that Xml into SQL (depending on what version of SQL there are different ways), but in the newer version's its a lot easier than it used to be:
http://www.codeproject.com/Articles/20847/Passing-Arrays-in-SQL-Parameters-using-XML-Data-Ty
Also, don't put any of that in ViewState; there's really no reason for that.
Storing an entire list of customers in ViewState is going to be hideously slow; storing all information for all customers in ViewState is going to be worse, unless your entire customer base is very very small, like about 30 records.
For a start, why are you loading all the customers into ViewState? If you have any significant number of customers, load the data a page at a time. That will at least reduce the amount of data flowing over the wire and might speed up your stored procedure as well.
In your position, I would focus on optimizing the data retrieval first (including minimizing the amount you return), and then worry about faster ways to store and display it. If you're up against unusual constraints that prevent this (very slow database; no profiling tools; not allowed to change stored procedures) than please let us know.
Solution 1: Include whatever criteria you need to filter on in your query, only return and render the requested records. No need to use viewstate.
Solution 2: Retrieve some reasonable page limit of customers, filter on the browser with javascript. Allow easy navigation to the next page.
I have a table full of id's,categories and weights that I need to reference in my program as I read in records that contain those categories. What is the most efficient method to read those from a database and put into a structure that I can reference?
The ID's (and possibly the names) would be unique
Data might look like:
ID,Category,Weight
1,Assignment,5
2,Test,10
3,Quiz,5
4,Review,3
Your best bet is to read in your table using a DataReader, and put each row into an object containing Category and Weight, then each object into a Dictionary.
If you're using a later version of .NET, you could always use Linq to just grab that data for you.
If you want to avoid a database hit to fetch static data, you can hard-code the values into a common class in your solution. A Dictionary collection would work fine here too.
The trade off of course is; 2 locations to manage for any possible future changes.