How to loop through a generic object in SSIS Script Task

How to loop through a generic object in SSIS Script Task - c#

I have a generic object that is passed into a script task from a SQL Process. The object will essentially be a data table but in order to get the full result set from the sql process i have to store it in a generic object.
So If i have:
Object A = Dts.Variables[0];
How then would I go about extracting and then manipulating its values.
Baseically what i want to do is:
Object A = Dts.Variables[0];
strin x = A.Column[0].value.tostring();
But this obviously won't work.

I could not get any of the above answers to work, so listed below is the code I used to load the datatable. "User::transactionalRepDBs" is a SSIS variable of Object (System.Object) that was loaded through a "Full result set" from a execute SQL task script. The script task used is C#. This link assisted me.
using System.Data.OleDb;
DataTable dt= new DataTable();
OleDbDataAdapter adapter = new OleDbDataAdapter();
adapter.Fill(dt, Dts.Variables["User::transactionalRepDBs"].Value);
String _showMe;
foreach (DataRow row in dt.Rows)
{
//insert what you want to do here
for (int i = 0, _showMe = ""; i < row.ItemArray.Length; i++ )
{
_showMe += row.ItemArray[i].ToString() + " | ";
}
MessageBox.Show("Data row #" + dt.Rows.IndexOf(row).ToString() + " value: " + _showMe);
}

There's nothing wrong with parsing a data table from an Object. I've seen Andy Leonard do it on his ETL frameworks.
You were on the correct path but you weren't seeing the whole picture. This code assigns the an object of type Variable (approximately) to A. You are then attempting to access a property that doesn't exist.
Object A = Dts.Variables[0];
Your need to grab the value of the variable. You can either do it as the assignment to A
Object A = Dts.Variables[0].Value;
Or if you needed to do something else with the actual variable, you'd keep your current code assignment of A then access the Value property.
Object A = Dts.Variables[0];
DataTable B = (DataTable) A.Value;
DataRow C = B.Row[0];
string x = C.Column[0].ToString();
The above code for datatable/datarow is approximate. The important take away is to access the goodies an SSIS variable is holding, you need to access the Value of the object.

Suggestion #1: Access variables by name, not numeric index.
Suggestion #2: Cast the result of the Value property to the type of object you're expecting.
Thus:
string myString = (string)Dts.Variables["MyString"].Value;
DataTable myTable = (DataTable)Dts.Variables["MyTable"].Value;
DataTable myOtherTable = Dts.Variables["MyOtherTable"].Value as DataTable;

awesome man,
This is working perfectly fine..
DataTable dt= new DataTable();
OleDbDataAdapter adapter = new OleDbDataAdapter();
adapter.Fill(dt, Dts.Variables["User::transactionalRepDBs"].Value);

// Works Perfectly fine ....ssis , c#
DataTable dt = new DataTable();
OleDbDataAdapter adapter = new OleDbDataAdapter();
adapter.Fill(dt, Dts.Variables["User::VariableObj"].Value);
foreach (DataColumn cols in dt.Columns)
{
MessageBox.Show("Colum Name = " + cols.ToString());
}
foreach (DataRow row in dt.Rows)
{
MessageBox.Show( "rows ID = " + row[0].ToString() + " rows
Name = " + row[1].ToString());
}

Related

How to use a SSIS Dataset inside Dataflow Script component for Source?

I'm looking for a way to use my Dataset variable in Script component of SSIS data flow task, here is what I tried till now
My First Task is Execute SQL task (CData PostgreSQL Task)
This would run a Postgre query and store the result in a variable User::resultSet which is of the data type DataSet
My Second Task is Data Flow Task
In my second task I wish to use the dataset as my source. I tried to achieve this by using Script Component (Source)
My Script
public override void CreateNewOutputRows()
{
DataTable dt = (DataSet)Variables.resultSet;
// The DataTable is now ready to use! No more recordset quirks.
foreach (DataRow dr in dt.Rows)
{
// Add a new output row for this transformed data.
OrdersBuffer.AddRow();
// Now populate the columns
OrdersBuffer.id = int.Parse(dr["id"].ToString());
OrdersBuffer.created = int.Parse(dr["created"].ToString());
OrdersBuffer.modified = int.Parse(dr["modified"].ToString());
}
}
Issue
In my Script, I cant figure out a way to convert the dataset into a datatable,
I believe the problem line is DataTable dt = (DataSet)Variables.resultSet; in my script
I also tried DataTable dt = resultSet.Tables[0]; & DataTable dt = resultSet.Table[0]; but all them are throwing syntax error.
Any lead or guidance will be really appreciated.

Try this one
DataTable dt = (Variables.resultSet as DataSet).Tables[0];
May throw an exception if type cast is not possible.

I have not used the dataset variable type but I have dont this with an OBJECT type variable and MS SQL DB. If you can change to use object variable this should work for you (or maybe if you cant you can see how I am passing the OBJECT variable and update your code to access the variable in similar way.
First I run a SQL task that has a result set as output in SSIS to my OBJECT variable data type. Then I pass it to my script task like this:
Inside the C# Script task I have this below code to access the OBJECT variable and then covert it to a data table that I use later on
Where this variable is my object variable type in the SSIS Package:
User::ObjDataToSaveToExportFile
// this gets the data object and sets ti to a data table
OleDbDataAdapter A = new OleDbDataAdapter();
System.Data.DataTable dt = new System.Data.DataTable();
A.Fill(dt, Dts.Variables["User::ObjDataToSaveToExportFile"].Value);
// for test data
//DataTable sourceTable = GetTestData();
DataTable sourceTable = dt;

You can use below approach of using DataAdapter to fill datatable. Reference Article
public override void CreateNewOutputRows()
{
// Set up the DataAdapter to extract the data, and the DataTable object to capture those results
OleDbDataAdapter da = new OleDbDataAdapter();
DataTable dt = new DataTable();
// Extract the data from the object variable into the table
da.Fill(dt, Variables.resultSet);
// Since we know the column metadata at design time, we simply need to iterate over each row in
// the DataTable, creating a new row in our Data Flow buffer for each
foreach (DataRow dr in dt.Rows)
{
// Add a new output row for this transformed data.
OrdersBuffer.AddRow();
// Now populate the columns
OrdersBuffer.id = int.Parse(dr["id"].ToString());
OrdersBuffer.created = int.Parse(dr["created"].ToString());
OrdersBuffer.modified = int.Parse(dr["modified"].ToString());
}

The name 'colType' does not exist in the current context

I am attempting to loop through a dataset's rows and columns in search for a match between the dataset's name column -- and the ColumnName from a DataReader object.
I have a new table called RECORDS which is empty at program startup. I also have a pre-populated table called ColumnPositions with a sub-set of column names found in the RECORDS table. This routine is intended to show a subset of all the available columns -- as a default display style.
My code works...except for the line of code that gets the dr["type"] value. I get the error:
The name 'colType' does not exist in the current context.
As you can clearly see, my string variables are declared outside the WHILE and FOREACH loops. The line statement colName = works just fine. But colType fails everytime. If I do a statement check in the Intermediate Window in VS2010 for ? dr["type"]" I get the result integer. But when I check ? colType, I get the above noted error message.
The intellisense for the DataRow object dr reveals an array of 6 items. Index 1 in the array maps to name. Index 2 maps to type. When I check the value of ? dr[2] in the Intermediate Window, the same result comes back integer. This is correct. But whenever this value is assigned to colType, VS2010 complains.
I'm no newbie to C# so I did a lot of testing and Googling before posting here. I'm hoping that this is a matter of me not seeing the forest through the trees.
Here's my code:
// get table information for RECORDS
SQLiteCommand tableInfo = new SQLiteCommand("PRAGMA table_info(Records)", m_cnCaseFile);
SQLiteDataAdapter adapter = new SQLiteDataAdapter(tableInfo);
DataSet ds = new DataSet();
adapter.Fill(ds);
DataTable dt = ds.Tables[0];
SQLiteCommand cmd = new SQLiteCommand("SELECT * FROM ColumnPositions WHERE ColumnStyle_ID = " + styleID + " ORDER BY ColumnPosition_ID ASC", m_cnCaseFile);
SQLiteDataReader colReader = cmd.ExecuteReader();
string colName = "";
string colType = "";
if (dt != null && colReader.HasRows)
{
while (colReader.Read())
{
foreach(DataRow dr in dt.Rows)
{
colType = Convert.ToString(dr["type"]);
colName = dr["name"].ToString();
if (colReader["ColumnName"].ToString() == colName)
{
DataGridViewColumn dgvCol = new DataGridViewColumn();
}
}
}
}
dt.Dispose();
colReader.Close();

Instead of using "dr["name"].ToString();", it is better to use "Convert.ToString(dr["name"]);"

Try using the array position instead of the column name:
colType = Convert.ToString(dr[1]);
and
colName = dr[0].ToString();
You probably don't need this, but here is the documentation for values returned by the SQLite PRAGMA table_info() command. LINK

C# code to find the end range of excel sheet data using Data tables

I am trying the find the Excel Row position to get the END point for my data fetch Range for excel using SSIS script component. I am using Datatable in a Script task to facilitate this.
In the below example my sheet range always starts with 1. I need to get the end position by searching "End of data" in the range A2:A10
Below is the Code I am using
string currentable ="Sheet1$"
string strSQL = "Select * From ["+currentTable+"A2:A10]";
int iCnt = 0;
OleDbDataAdapter objAdapter = new OleDbDataAdapter(strSQL, excelConnection);
DataSet ds = new DataSet();
objAdapter.Fill(ds, currentTable);
DataTable dt = ds.Tables[currentTable];
foreach (DataRow row in dt.Rows)
{
MessageBox.Show("In for Loop");
iCnt = iCnt + 1;
MessageBox.Show(row[iCnt].ToString());
}
In this code I am getting the following error
System.IndexOutOfRangeException: Cannot find column 1.
I need to browse through the First column of the excel and search the string "End of Data" and get its position.
Any ideas/help
Thanks

It looks like there's a bit of an error in the code above. The SQL used to select data selects a single row, but the for loop looks at the second column (iCnt + 1) when only one column is present (row[0]).
Regardless, something like this might work for your needs:
var lastRow = dt.Rows.AsEnumerable().Select((row, index) => new { Name = row.ToString(), index }).First(row => row.Name == "End of data").index;

Copy rows from one Datatable to another DataTable?

How can I copy specific rows from DataTable to another Datable in c#? There will be more than one row.

foreach (DataRow dr in dataTable1.Rows) {
if (/* some condition */)
dataTable2.Rows.Add(dr.ItemArray);
}
The above example assumes that dataTable1 and dataTable2 have the same number, type and order of columns.

Copy Specified Rows from Table to another
// here dttablenew is a new Table and dttableOld is table Which having the data
dttableNew = dttableOld.Clone();
foreach (DataRow drtableOld in dttableOld.Rows)
{
if (/*put some Condition */)
{
dtTableNew.ImportRow(drtableOld);
}
}

Try This
String matchString="ID0001"//assuming we have to find rows having key=ID0001
DataTable dtTarget = new DataTable();
dtTarget = dtSource.Clone();
DataRow[] rowsToCopy;
rowsToCopy = dtSource.Select("key='" + matchString + "'");
foreach (DataRow temp in rowsToCopy)
{
dtTarget.ImportRow(temp);
}

Check this out, you may like it (previously, please, clone table1 to table2):
table1.AsEnumerable().Take(recodCount).CopyToDataTable(table2,LoadOption.OverwriteChanges);
Or:
table1.AsEnumerable().Where ( yourcondition ) .CopyToDataTable(table2,LoadOption.OverwriteChanges);

Supported in: 4, 3.5 SP1, you can now just call a method on the object.
DataTable dataTable2 = dataTable1.Copy()

As a result of the other posts, this is the shortest I could get:
DataTable destTable = sourceTable.Clone();
sourceTable.AsEnumerable().Where(row => /* condition */ ).ToList().ForEach(row => destTable.ImportRow(row));

I've created an easy way to do this issue
DataTable newTable = oldtable.Clone();
for (int i = 0; i < oldtable.Rows.Count; i++)
{
DataRow drNew = newTable.NewRow();
drNew.ItemArray = oldtable.Rows[i].ItemArray;
newTable.Rows.Add(drNew);
}

I needed to copy rows from multiple tables with the same structure into a new table to be used as a datasource for datagridview:
// Generate DataTable[] alltables from multiple datatables
DataTable newTable = alltables[0].Clone();
foreach (DataTable dt in alltables)
{
for (int i = 0; i < dt.Rows.Count; i++)
newTable.Rows.Add(dt.Rows[i].ItemArray);
}

below sample would be the fastest way to copy one row.
each cell is being copied based on the column name.
in case you dont need a specific cell to copy then have a try catch or add if.
if your going to copy more than 1 row then loop the code below.
DataRow dr = dataset1.Tables[0].NewRow();
for (int i = 0; i < dataset1.Tables[1].Columns.Count; i++)
{
dr[dataset1.Tables[1].Columns[i].ColumnName] = dataset1.Tables[1].Rows[0][i];
}
datasetReport.Tables[0].Rows.Add(dr);
dataset1.Tables[1].Rows[0][i]; change the index 0 to your specified row index or you can use a variable if your going to loop or if its going to be logical

private void CopyDataTable(DataTable table){
// Create an object variable for the copy.
DataTable copyDataTable;
copyDataTable = table.Copy();
// Insert code to work with the copy.
}

To copy whole datatable just do this:
DataGridView sourceGrid = this.dataGridView1;
DataGridView targetGrid = this.dataGridView2;
targetGrid.DataSource = sourceGrid.DataSource;

For those who want single command SQL query for that:
INSERT INTO TABLE002
(COL001_MEM_ID, COL002_MEM_NAME, COL002_MEM_ADD, COL002_CREATE_USER_C, COL002_CREATE_S)
SELECT COL001_MEM_ID, COL001_MEM_NAME, COL001_MEM_ADD, COL001_CREATE_USER_C, COL001_CREATE_S
FROM TABLE001;
This query will copy data from TABLE001 to TABLE002 and we assume that both columns had different column names.
Column names are mapped one-to-one like:
COL001_MEM_ID -> COL001_MEM_ID
COL001_MEM_NAME -> COL002_MEM_NAME
COL001_MEM_ADD -> COL002_MEM_ADD
COL001_CREATE_USER_C -> COL002_CREATE_USER_C
COL002_CREATE_S -> COL002_CREATE_S
You can also specify where clause, if you need some condition.

There is better way to do this.
DataTable targetDataTable = new DataTable();
targetDataTable = changedColumnMetadata.AsEnumerable().Where(dataRow => entityName.Equals(dataRow["EntityName"])).CopyToDataTable();
Please try this and let me know in case of any issues.

You can do it calling the DataTable.Copy() method, for example:
DataSet ds = new DataSet();
System.Data.DataTable dt = new System.Data.DataTable();
dt = _BOSearchView.DS.Tables[BusLib.TPV.TableName.SearchView].Copy();
ds.Tables.Add(dt);
UltGrdSaleExcel.SetDataBinding(ds, dt.TableName, true);

use Merge, this will append all rows of newDataTable with oldDateTale
oldDateTale.Merge(newDataTable);

DataSets - Class Model - How to get Bool Value from a DataSet

I am new to C# and ASP.net. I am a ColdFusion Programmer but I am catching on to ASP.net.
I will give details then ask my question....
I have been able to get it where I call from the Code Behind a Service Layer Class File (or Bussiness Logic Layer) then in that one call a Data Access Layer Class file. I have been sending back a DataSet from the Data Access Layer then in the Code Behind moving the code to a table so I can read it row by row.
I was looking for a way to get the Boolean values when I am on the Edit screen to set the check boxes to checked when I found this post.
C# Assigning a value to DataRow["haswhatnots"] = hasWhatnots is awful slow
I see a mention that they should...
Alternatively - use a class model instead of a DataTable that sounds great if it is what I think it is.
What is the Best way to do a CRUD screens where you use the Class file to pull back the data and what format should the Data be in?
I am used to in ColdFusion data coming back in a QuerySet and then we can loop through it. We can convert he query to Array of Structures or and Array of Obects, XML or JSon but I am still running into issues understanding HOW to get the data out of the DataSet when I get it back. I get String Data fine right now but maybe there is a very simple example with maybe one page...
I want to figure out how to get the Boolean values right now is my current issue.
Thanks for any help,
Nathan
P.S. if there is a better more professional way to do this let me know. I want to keep using a (Presentation/CodeBehind/Service/Data Access) type of layers.
Protected void Page_Load(Object sender, EventArgs e)
{
if(!IsPostBack)
{
DataTable dt = new DataTable();
DataRow dr;
PageService myPage = new PageService();
DataSet ds = myPage.getPages();
int rowCount = ds.Tables[0].Rows.Count;
ds.Columns.Add(new DataColumn("PageID", typeof(Int32)));
ds.Columns.Add(new DataColumn("Name", typeof(String)));
ds.Columns.Add(new DataColumn("PageHTMLContent", typeof(String)));
ds.Columns.Add(new DataColumn("NavigationText", typeof(String)));
ds.Columns.Add(new DataColumn("TopMenu", typeof(Boolean)));
ds.Columns.Add(new DataColumn("SubMenu", typeof(Boolean)));
ds.Columns.Add(new DataColumn("DisplayOrder", typeof(Int32)));
ds.Columns.Add(new DataColumn("Active", typeof(Boolean)));
for (int i = 0; i < rowCount; i++)
{
dr = dt.NewRow();
dr[0] = i;
dr[1] = i.ToString;
dr[2] = i.ToString;
dr[3] = i.ToString;
dr[4] = i; // Not sure what to do here for the Boolean Value
dr[5] = i; // Not sure what to do here for the Boolean Value
dr[6] = i;
dr[7] = i; // Not sure what to do here for the Boolean Value
}
ds.Tables.Add(dt);
foreach (DataRow dataRow in ds.Tables[0].Rows)
{
PageIDHiddenField.Value = dataRow["PageID"];
NameTextBox.Text = dataRow["Name"].ToString();
PageHTMLContentTextBox.Text = dataRow["PageHTMLContent"];
NavigationTextTextBox.Text = dataRow["NavigationText"];
TopMenuCheckBox.Checked = dataRow["TopMenu"]; // Not sure what to do here for the Boolean Value
SubMenuCheckBox.Checked = dataRow["SubMenu"]; // Not sure what to do here for the Boolean Value
DisplayOrder.Text = dataRow["DisplayOrder"].ToString();
ActiveCheckBox.Checked = dataRow["Active"]; // Not sure what to do here for the Boolean Value
}
}
}

You need to cast the value to the correct type. Because dataRow's indexer property will return objects of type object, you need to cast it to the right type. Therefore, you need to cast it to Boolean like so:
TopMenuCheckBox.Checked = (bool)dataRow["TopMenu"];
To do one loop to iterate over the rows in the DataSet you'd do something like:
DataSet ds = myPage.getPages();
foreach (DataRow row in ds.Tables[0].Rows)
{
PageIDHiddenField.Value = row["PageID"];
NameTextBox.Text = (string)row["Name"];
...
TopMenuCheckBox.Checked = (bool)row["TopMenu"];
}
You can use DataRow's indexer property to get the specific column.

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

How to loop through a generic object in SSIS Script Task - c#

awesome man, This is working perfectly fine.. DataTable dt= new DataTable(); OleDbDataAdapter adapter = new OleDbDataAdapter(); adapter.Fill(dt, Dts.Variables["User::transactionalRepDBs"].Value);

Related

How to use a SSIS Dataset inside Dataflow Script component for Source?

The name 'colType' does not exist in the current context

C# code to find the end range of excel sheet data using Data tables

Copy rows from one Datatable to another DataTable?

DataSets - Class Model - How to get Bool Value from a DataSet

Categories

Resources