This is my first post on here. I'm attempting to create a 'simple' charting program (windows form based in c#). I say simple because I'm currently only playing around with only 2 series maximum, and a few transformations (percent changes, actual changes, moving averages and moving sums). It will get more complicated than this but having the limited functionality first might help me get a better handle on how all of this works in C#.
I've been searching on and off for a couple of days now but have had no luck so far with my specific situation. I'll try to be as detailed as possible. The user retrieves the time series data from a SQL Server. This part of the program is behaving as expected. I'm creating 2 queries (one for each series) to retrieve the data separately. I do the transformations in the SQL query. Each comes via a SQL adapter which is then placed into a data table. The series may be of different frequencies and the dates may not overlap (i.e. sometimes a stock prices will be daily, and exports will be monthly, or GDP quarterly). The exports number may come in as the first of every month, while stock prices may be missing a value on this date if it was a weekend. I suspect this part is important for my issue.
Nonetheless, I double checked this step and everything works as expected (the values are all coming in correctly at the right dates).
To add those series to a chart, I merge the two data tables like so (I believe this is where my issues are coming from):
myTable = myTables1.Copy();
myTables1.PrimaryKey = new DataColumn[] { myTables1.Columns["dates"] };
myTables2.PrimaryKey = new DataColumn[] { myTables2.Columns["dates"] };
myTable.Merge(myTables2, true);
Datagridview of the Merged data table here looks good to me
Then I create the chart: I've tried two different methods here. The first is to create each series in a 2 step loop (one for each series) and loop through each row in the table and add an x and y value for each series. I've also tried to set the data source for the chart to be the table, and create 2 series and set the X and Y members as the names of the columns in the table.
Like so:
mychart.DataSource = myTable;
Then in a loop for each series:
mychart.Series.Add(seriesname);
mychart.Series[seriesname].XValueMember = "dates";
mychart.Series[seriesname].YValueMembers = seriesname;
Regardless, my second series is always a bit off. Either there are straight lines going across or it misses some values (and by adding a tool tip i can tell that the dates where one series may have a value while the other does not, is where the problems are occurring).
I'm not looking for help with syntax (just ideas). So my question is: is there a standard or preferred way of getting and plotting series with different frequencies (or which may have different x values)? Alternatively, is there a good source/documentation where i can read up on this? I would add that i have a similar program using visual basic which uses one SQL query regardless of how many series there are. This works in terms of the chart looking as I'd expect it to, but it makes transformations much more complicated given the randomness of the null or empty values in the final table.
Related
TL;DR In a SSRS 2008 report which uses a custom assembly to do some extra calculations can I pass an entire report dataset as a method parameter?
Full story
I have an SSRS report with 3 datasets, each returned from an SQL query.
(In case it makes a difference to my question they're currently shared datasets although I'm sure local would work too)
The largest and primary dataset is a list of tasks which may or may not have been completed. I have information in here such as the ID, status, create date/time, target resolution hours etc of each task.
This dataset is displayed in a tablix and is the focus of the report.
The remaining two datasets are not displayed and are for reference. One is a simple one column query which returns a list of holiday dates for the UK. The other is a small table which contains our exact business hours.
At the moment I'm able to loop through the rows in the tablix of tasks and pass multiple values from the current row to a method. This is useful if I want to do some calculations based on data found only in the current row. For example I could take the create date/time and the response target hours and the assembly would return a target date/time for the current task. Cool so far.
I want to do a more complicated version of this where I not only pass in the row data but the 2 other datasets to get my return value. This is because in reality the due date calculation is much more complex and must take into account changing business hours and holidays from the other 2 datasets.
Can I pass a dataset as a method parameter to an assembly? Something like:
=Code.MyClass.MyMethod(val1, val2, dataset1, dataset2);.
I've been unable to find much definitive information on this. Nearly all tutorials demonstrate what I'm already doing by processing single rows. I'm sure I had an MSDN article that hinted this was not possible but I've lost it (helpful I know). There's a post on the Microsoft forums where a moderator says it's not possible. The general lack of information and tutorials suggests it's not possible or I'm doing this in the wrong way.
Any suggestions?
(I have alternate solutions such as having the assembly fetch the other datasets or just writing something outside SSRS but I'm not pursuing those until I knnow whether it can be done this way).
An older topic on the MSDN forums Iterate through rows of dataset in report's custom code offers a more definitive answer as well as a potential solution to this problem.
Passing the DataSet as an object or collection is not a possibility because:
A dataset in Reporting Services is not the same type of object as an ADO.Net dataset. A report dataset is an internal object managed by the SSRS runtime (it's actually derived from a DataReader object) and not an XML structure containing datatables, etc. and cannot be passed into the report's custom code.
The only way to effectively loop through the rows of a report dataset is to call a custom function or referenced method in a report data region expression. Using this technique, it may be possible to pass all of the the row and field information into a code structure, array or collection.
The hint given in the above statement suggests passing row and field information into a code structure. A contributor to the linked MSDN topic, Migeul Catalao developed a workaround using such an approach.
A real-world scenario of it's usage with example code demonstrating Migeul Catalao's solution can be found here.
Granted, it is still more of a row-by-row approach, so I would strongly suggest moving outside of SSRS and pursue alternative solutions.
Although I've accepted the other answer due to it being clear and helpful I didn't use that solution in the end (I was too stupid to understand it) and went for something else that works.
Disclaimer: This is a horrible hack. It works absolutely great in my scenario so I though I'd share in case it was useful to somebody else. There are many pitfalls here which could most likely be worked around given time.
I ended up following the advice in the comment given by Steven White and looking into LookupSet. This function allows you to query a dataset to return matching rows and a single column of data.
It looks like this:
LookupSet(Fields!ComparisonField.Value, // The value to search for, e.g '001'.
Fields!MatchField.Value, // The column to match on in the target dataset.
Fields!MyColumn.Value, // The column that will be returned.
"MyDataSet") // The dataset to search.
This returns a string array representing the returned values.
So far so good, but I needed ALL columns and rows. This is where the dirty hack appears in the form of string concatenation:
LookupSet(0, // Dummy ID 0.
0, // Matches the dummy ID 0 so all rows are returned.
Fields!Column1.Value + "[^]" // I concatenate all of the values into
+ Fields!Column2.Value + "[^]" // one string with the separator [^]
+ Fields!.Column3.Value, // so I can split them later.
"MyDataSet") // The dataset to query
I can now pass this to my custom assembly:
=MyAssemblyNamespace.Class.Method(LookupSet(0,0,Fields!Column1.Value..., "MyDataSet"), other, parameters, here)
Now in my C# method I have a generic object which after some reflection is actually an array of strings.
Cast to something useful:
var stringList = ((IEnumerable)MyDataSetObject).Cast<string>().ToList();
Split it:
foreach (var item in stringList)
{
var columns = item.Split(new[] { "[^]" }, StringSplitOptions.None);
// columns is a string[] which holds each column value for the current row
// So columns[0] is the value for column 1 in this row
// In my case I pushed the values to a DataTable row each time and built a datatable
// which when finished represented my dataset in full with all rows and columns.
}
I hope this makes sense to anyone trying to achieve a similar result.
I have made a chart and added the points on runtime. The data is plotted as
Chart1.ChartAreas(0).AxisX.IntervalType = DateTimeIntervalType.Weeks
Chart1.Series("PH").Points.AddXY(DateTime.Parse(dr.Item("readtime")).ToString("MM-dd"), dr.Item("ph"))
The data that is plotted holds 7 days of data.
As shown below, the date is repeated.
How to achieve the x axis showing only 11-08, 11-13, 11-14 in this case?
Here is a sample data
When you get data from database, use like this,
SELECT SUM(ph),SUM(tmp),SUM(orp),SUM(sal),SUM(ec),SUM(tds),SUM(do),readtime
FROM <your tables>
GROUP BY readtime
The results set returned from this query is the thing you are looking for....
The data that you show is obviously invalid data.
You have several zero readings occurring at exactly the same time - midnight.
I suspect these are not even real readings.
You need to exclude them (and you should probably find out why this is happening - maybe a parsing error?)
Please note that the database design I have now is fully in sandbox mode. Nothing is finalized. Everything (again this is in sandbox mode) is in one single table. Also, I'm in now way looking for coding help. I'm looking for the right theoretical/logical approach to this puzzle so I can go in and play with the coding myself. I learn better that way. If I need coding help, I'll come back here for further assistance.
I'm in the process of creating the first of a few CheckBoxList for a manually created form submittal. I've been looking over multiple ways to not only create said CheckBoxList but to enter it into the database. However, the challenge I'm facing (mainly because I haven't encountered this scenario before and I'd like to learn this) is that I not only need to worry about entering the items correctly in the database but I will eventually need to produce a report or make a printable form from these entries.
Let's say I have an order form for a specific type of Barbeque grill and I will need to send this form out to distriution centers across the nation. The distribution centers will need to pull said barbecues if they are highlighted on the form.
Here's what the CheckBoxList for the distibution centers will look like:
All
Dallas
Miami
Los Angeles
Seattle
New York
Chicago
Phoenix
Montreal
If the specific city (or all the cities) are checked, then the distribution center will pull the barbecue grill for shipment.
The added part is that I want to:
be able to create a grid view from this database for reporting to note which distribution center got orders for barbecues and
be able to create reports to tell what distribution center sent out barbecue orders in a given month (among other reporting).
Here's what I'm playing around with right now.
In my aspx page I have a checkboxlist programmed with all the distribution centers entered as a listitem as well as an option for 'ALL' (of the distribution centers).
I also created a dedicated column in this table that holds all the information in the listitem and programmed a sqldataconnection to this table to play with the programmability of leveraging the database for this purpose.
When it comes to writing the selections to the database, I originally created a column for each destination city including the 'All' option. I was toying around with just putting the selections into one single column but with some of the information I've been reading today about Database Normalization, the former options seems to be a better one than the latter. Is this correct practice for situations such as this especially if I need to think about reporting? Do I put the CheckBoxList data in one cell in a specific column? Do I create seprate columns for each distribution center? Am I even on the right track here?
Depending on the amount of cities that you want to store, I've used bitwise operators to store small amounts of data. Effectively, it would store it in the table like this:
CityID Location
2 Dallas
4 Miami
8 New York
16 Chicago
32 Montreal
and keep going in base 2 for additional cities.
When your user selects multiple cities for the order, the value that gets inserted into the database for cities is a bitwise OR calculation. So if they select Dallas, New York, and Chicago, you would be doing the following:
2 OR 8 OR 16
Which would equal 26
Now, you can use bitwise AND on the resulting value. So if checking for Miami the following is the evaluation:
26 AND 4 = 0
which indicates that Miami was not selected. Any value that was selected in the evaluation, it would return its ID like this:
26 AND 8 = 8
Again, I've only used this for small subsets of data, and to make the data storage as compact as possible. Computationally, it may be a trifle more expensive that some other methods, but I'm not 100% certain.
Note: This might not be the best approaches but I have seen them used.
1) Having one column of comma-delimited string
This should work well if the options don't have IDs in the database (having a separate referenced table)
You will need to loop through the checkbox list, obtained the selected options and concatenate them with String.Join()
You will need to split the string upon receiving it from the db and use it to check the checkboxes if there text is found in the resulting array
Problem: You might need a split function in the DB that converts the comma-separated string into rows. There split function implementation on the web/stackoverflow
2) You can have a separate table for the locations e.g. xxxxtable_location where the FK to the main table is referenced. This will be a one-many table
ParentID, Location
1 Dallas
2 Miami
2 New York
2 Chicago
3 Miami
I'm writing an application that I will use to keep up with my monthly budget. This will be a C# .NET 4.0 Winforms application.
Essentially my budget will be a matrix of data if you look at it visually. The columns are the "dates" at which that budget item will be spent. For example, I have 1 column for every Friday in the month. The Y axis is the name of the budget item (Car payment, house payment, eating out, etc). There are also categories, which are used to group the budget item names that are similar. For example, a category called "Housing" would have budget items called Mortgage, Rent, Electricity, Home Insurance, etc.
I need a good way to store this data from a code design perspective. Basically I've thought of two approaches:
One, I can have a "BudgetItem" class that has a "Category", "Value", and "Date". I would have a linear list of these items and each time I wanted to find a value by either date or category, I iterate this list in some form or fashion to find the value. I could probably use LINQ for this.
Second, I could use a 2D array which is indexed first by column (date) and second by row. I'd have to maintain categories and budget item names in a separate list and join the data together when I do my lookups somehow.
What is the best way to store this data in code? I'm leaning more towards the first solution but I wanted to see what you guys think. Later on when I implement my data persistence, I want to be able to persist this data to SQL server OR to an XML file (one file per monthly budget).
While your first attempt looks nicer, obviusly the second could be faster (depends on how you implement it). However when we are talking about desktop applications which are not performance critical, your first idea is definitely better, expecially because will help you a lot talking about maintaining your code. Also remember that the entity framework could be really nice in this situation
Finally if you know how to works with XML, I think is really better for this type of project. A database is required only when you have a fair amount of tables, as you explained you will only have 2 tables (budgetitem and category), I don't think you need a database for such a simple thing
Scenario:
I have an application (C#) that expects a SQL database and login, which are set by a user. Once connected, it checks for the existence of several table and creates them if not found.
I'd like to expand on this by having the program be capable of adding columns to those tables if I release a new version of the program which relies upon the new columns.
Question:
What is the best way to programatically check the structure of an existing SQL table and create or update it to match an expected structure?
I am planning to iterate through the list of required columns and alter the existing table whenever it does not contain the new column. I can't help but wonder if there's an approach that is different or better.
Criteria:
Here are some of my expectations and self-imposed rules:
Newer versions of the program might no longer use certain columns, but they would be retained for data logging purposes. In other words, no columns will be removed.
Existing data in the table must be preserved, so the table cannot simply be dropped and recreated.
In all cases, newly added columns would allow null data, so the population of old records is taken care of by having default null values.
Example:
Here is a sample table (because visual examples help!):
id datetime sensor_name sensor_status x1 x2 x3 x4
1 20100513T151907 na019 OK 0.01 0.21 1.41 1.22
2 20100513T152907 na019 OK 0.02 0.23 1.45 1.52
Then, in a new version, I may want to add the column x5. The "x-columns" are all data-storage columns that accept null.
Edit:
I updated the sample table above. It is more of a log and not a parent table. So the sensors will repeatedly show up in this logging table with the values logged. A separate parent table contains the geographic and other logistical information about the sensor, making the table I wish to modify a child table.
This is a very troublesome feature that you're thinking about implementing. i would advise against it and instead consider scripting changes using a 3rd party tool such as Red Gate's Sql Compare: http://www.red-gate.com/products/SQL_Compare/index.htm
If you're in doubt, consider downloading the trial version of the software and performing a structure diff script on two databases with some non-trivial differences. You'll see from the result that the considerations for such operations are far from simple.
The other way around this type of issue is to redesign your database using the EAV model: http://en.wikipedia.org/wiki/Entity-attribute-value_model (Pivots to dynamically add rows thus never changing the structure. It has its own issues but it's very flexible.)
(To utilize a diff tool you would have to have a copy of all of your db versions and create diff scripts which would go out and get executed with new releases and upgrades. That's a huge mess of its own to maintain. EAV is the way for a thing like this. It wrongfully gets a lot of flak for not being as performant as a traditional db structure but i've used it a number of times with great success. In fact, i have an HIPAA-compliant EAV db (Sql Server 2000) that's been in production for over six years with several of the EAV tables containing tens or millions of rows and it's still going strong w/ no big slow down. Of course we don't do heavy reporting against that db. For reports we have an export that flattens the data into a relational structure.)
The common solution i see would be to store in your database somewhere version information. maybe have a really small table:
CREATE TABLE DB_PROPERTIES (key varchar(100), value varchar(100));
then you could add a row:
key | value
version | 12
Then you could just create a sql update script (or set of scripts) which updates the db from version 12 to version13.
declare v varchar(100)
select v=value from DB_PROPERTIES where key='version'
if v ='12'
#do upgrade from 12 to 13
elsif v='11'
#do upgrade from 11 to 13
...and so on
depending on what upgrade paths you wanted to support you could add more cases. You could also obviously move this upgrade logic into C# and or whatever design works for you. But having the db version information stored in the database will make it much easier to figure out what is already there, rather than querying for all the db structures individually.
If you have to build something in such a way as to rely on the application making table changes, your design is flawed. You should have a related table for the sensor values (x1, x2, etc.), then you can just add another record rather than having to create a new column.
Suggested child table structure
READINGS
ID int
Reading_type varchar (10)
Reading_Value int
Then data in the table would read:
ID Reading_type Reading_value
1 x1 2
1 x2 3
1 x3 1
2 x1 7
Try Microsoft.SqlServer.Management.Smo
These are a set of C# classes that provide an API to SQL Server database objects.
The Microsoft.SqlServer.Management.Smo.Table has a Columns Collection that will allow you to query and manipulate the columns.
Have fun.