Develop Reference

Extract Contents of DataSet into an Object

I have this DataSet which calls my Stored Procedure and returns a list of integers. How can I extract the list of integers which I could store in a variable be it a collection which grows in size such as a List<T> or a primitive data type like an array of integers.
Below is my code:
private DataSet getSubGroupsBelongingToUser()
{
DataTable variable;
DataSet DS;
myConnectionString = ConfigurationManager.ConnectionStrings["FSK_ServiceMonitor_Users_Management.Properties.Settings.FSK_ServiceMonitorConnectionString"].ConnectionString;
using (mySQLConnection = new SqlConnection(myConnectionString))
{
SqlParameter param = new SqlParameter("#UserId", getUserID(cbxSelectUser.Text));
DS = GetData("Test", param);
variable = DS.Tables[0];
}
return DS;
}
When I hover over the DS magnifier (refer to image):
I want to retrieve and store that list of integers somewhere. How can I go about doing this? All the examples I came across online make use of linq and that is not applicable here since I am getting the results from my stored procedure which requires one input parameter. Here is the definition of the stored procedure below:
create proc [dbo].[Test]
#UserId smallint
as
begin
select DepartmentSubGroupId from DepartmentSubGroupUser
where UserId= #UserId
end
GO
So essentially when you pass in a UserId, you should get those values. I am using SQL Server as my DBMS.

The simplest and most efficient approach would be to not use a DataSet/DataTable at all:
private List<int> GetSubGroupsBelongingToUser()
{
List<int> list = new List<int>();
using (var con = new SqlConnection(ConfigurationManager.ConnectionStrings["FSK_ServiceMonitor_Users_Management.Properties.Settings.FSK_ServiceMonitorConnectionString"].ConnectionString))
using (var cmd = new SqlCommand("Test", con))
{
cmd.CommandType = CommandType.StoredProcedure;
var param = new SqlParameter("#UserId", SqlDbType.Int).Value = int.Parse(cbxSelectUser.Text);
cmd.Parameters.Add(param);
con.Open();
using (var rd = cmd.ExecuteReader())
{
while (rd.Read()) list.Add(rd.GetInt32(0));
}
} // no need to close the connection with the using
return list;
}
If you insist on the DataTable, at least it's more conscise:
return DS.Tables[0].AsEnumerable().Select(r => r.Field<int>(0)).ToList();

As #David pointed out the simplest option would be to use a SqlDataReader and loop through all the records.
However, if your heart is set on DataTables then all you need to do is to iterate all rows from the result table, grab the value from column DepartmentSubGroupId and add it to a list. You can do that with Linq also like this:
return DS.Tables[0].Rows
.Cast<DataRow>() // Rows is an ICollection and you need to cast each item
.Select(r => (int)r["DepartmentSubGroupId"]) // For each row get the value from column DepartmentSubGroupId
.ToList();

Before I saw #Tim's solution, I had already prepared this (which works just as well as most of the above solutions):
public List<int> getSubGroupsBelongingToUser()
{
List<int> DepartmentSubGroupIds = new List<int>();
myConnectionString = ConfigurationManager.ConnectionStrings["FSK_ServiceMonitor_Users_Management.Properties.Settings.FSK_ServiceMonitorConnectionString"].ConnectionString;
using (mySQLConnection = new SqlConnection(myConnectionString))
{
SqlParameter parameter = new SqlParameter("#UserId", getUserID(cbxSelectUser.Text));
mySQLCommand = new SqlCommand("Test", mySQLConnection);
mySQLCommand.CommandType = CommandType.StoredProcedure;
mySQLCommand.Parameters.Add(parameter);
mySQLConnection.ConnectionString = myConnectionString;
mySQLConnection.Open();
SqlDataReader sqlDataReader = mySQLCommand.ExecuteReader();
while (sqlDataReader.Read())
{
DepartmentSubGroupIds.Add(Convert.ToInt32(sqlDataReader["DepartmentSubGroupId"]));
}
}
return DepartmentSubGroupIds;
}
Thanks everybody, much appreciated.

Make Data columns from a sql adapter

Is there a way, where I can use a sql adapter and convert the sql query results into a Data Columns? I'm kinda new a datatables. I need to build this dynamically since my column names are stored into a sql table. I keep running into datarows not columns. What I have built so far:
string feedcolumns = #"select FeedColumnName from myTable where FeedProcessID = #feedprocessid";
SqlCommand columnscommand = new SqlCommand(feedcolumns, connUpd);
DataTable dt = new DataTable("datafeed");
foreach(DataColumn dc in dt.Columns)
{
dc = new
dt.Columns.Add(dc);
}

You can fill a DataTable directly from an SqlReader, no need to go column by column.
One thing I did notice was that your SQL statement had a parameter in it that was never assigned to the command object, so I added it in
string feedcolumns = "select FeedColumnName from myTable where FeedProcessID = #feedprocessid";
DataTable dt = new DataTable("datafeed");
using (SqlConnection connUpd = new SqlConnection("MyConnection")) {
using (SqlCommand columnscommand = new SqlCommand(feedcolumns, connUpd)) {
columnscommand.Parameters.AddWithValue("#feedprocessid", feedprocessid);
connUpd.Open();
var dataReader = columnscommand.ExecuteReader();
dt.Load(dataReader);
}
}

Getting data from Database and storing into Dictionary

I am trying to get the data from a MySQL table and storing it into a dictionary. I know I can do that with a loop but DB table contains more than a million tuples and will slow the progress. Is there any way to do that without having to loop all the entries?
DB table has two columns, 1st(key) float type and 2nd(value) varchar.
My Current approach:
public static Dictionary<Single, string> GetData()
{
Dictionary<Single, string> dic= new Dictionary<Single, string>();
string query = "select * from table;";
if (OpenConnection() == true)
{
try
{
MySqlCommand cmd = new MySqlCommand(query, connection);
MySqlDataReader reader = cmd.ExecuteReader();
if (reader.Read())
{
//Something to store DB columns into dictionary without having to loop.
}
reader.Close();
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
return dic;
}

Next is what you want:
try
{
MySqlCommand cmd = new MySqlCommand(query, connection);
MySqlDataAdapter msda = new MySqlDataAdapter(cmd);
DataSet ds = new DataSet();
msda.Fill(ds);
your_control.DataSource = ds;
your_control.DataBind();
//your control means some repeater, or gridview, or anything else, which may show all your elements together.
}
But you also should consider that if you have millions of records, it would be better to make them selected partially. For example, 100 records per page, using row_number or similar methods

get data from variable table and return as datatable c#

I need to get retrive all of the data from specified tables and I don't need the data to be strongly typed so I am returning it as a data table.
public DataTable GetByTypeName(String t)
{
var type = Type.GetType(t);
var dt = new DataTable();
using (var sqlConn = new SqlConnection(ConfigurationManager.ConnectionStrings["MasterPlanConnectionString"].ConnectionString))
{
var sqlComm = new SqlCommand("SELECT * FROM #table", sqlConn);
sqlComm.Parameters.AddWithValue("#table", type.Name);
sqlConn.Open();
var dr = sqlComm.ExecuteReader(CommandBehavior.CloseConnection);
dt.Load(dr);
}
return dt;
}
When I run this I get the error
System.Data.SqlClient.SqlException was unhandled by user code
Message=Must declare the table variable "#table".
I cannot figure out why this isn't working as I have declared #table. I know this method is open to some bad sql attacks so I plan to add in some protection about exactly what types can be queried against.

You can construct your query dynamically - (should be ok over here, but may expose your query to sql injection)
var query = String.Fromat("Select * from [{0}]", type.Name);
var sqlComm = new SqlCommand(query, sqlConn);
/*sqlComm.Parameters.AddWithValue("#table", type.Name);*/

Skip some columns in SqlBulkCopy

I'm using SqlBulkCopy against two SQL Server 2008 with different sets of columns (going to move some data from prod server to dev). So want to skip some columns not yet existed / not yet removed.
How can I do that? Some trick with ColumnMappings?
Edit:
I do next:
DataTable table = new DataTable();
using (var adapter = new SqlDataAdapter(sourceCommand))
{
adapter.Fill(table);
}
table.Columns
.OfType<DataColumn>()
.ForEach(c => bulk.ColumnMappings.Add(
new SqlBulkCopyColumnMapping(c.ColumnName, c.ColumnName)));
bulk.WriteToServer(table)
and get:
The given ColumnMapping does not match up with any column in the source or destination.

DataTable table = new DataTable();
using (var adapter = new SqlDataAdapter(sourceCommand))
{
adapter.Fill(table);
}
using (SqlBulkCopy bulk = new SqlBulkCopy(targetConnection, SqlBulkCopyOptions.KeepIdentity, null) { DestinationTableName = tableName })
{
foreach (string columnName in GetMapping(stringSource, stringTarget, tableName))
{
bulk.ColumnMappings.Add(new SqlBulkCopyColumnMapping(columnName, columnName));
}
targetConnection.Open();
bulk.WriteToServer(table);
}
private static IEnumerable<string> GetMapping(string stringSource, string stringTarget, string tableName)
{
return Enumerable.Intersect(
GetSchema(stringSource, tableName),
GetSchema(stringTarget, tableName),
StringComparer.Ordinal); // or StringComparer.OrdinalIgnoreCase
}
private static IEnumerable<string> GetSchema(string connectionString, string tableName)
{
using (SqlConnection connection = new SqlConnection(connectionString))
using (SqlCommand command = connection.CreateCommand())
{
command.CommandText = "sp_Columns";
command.CommandType = CommandType.StoredProcedure;
command.Parameters.Add("#table_name", SqlDbType.NVarChar, 384).Value = tableName;
connection.Open();
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
yield return (string)reader["column_name"];
}
}
}
}

When SqlBulkCopyColumnMapping is used, only columns for which mappings are created will be copied.
If you do not create a mapping for a column, it will be ignored by the copy process.
You can see this in the demo code here - the sample source table in the AdventureWorks demo database contains more columns than are mapped or copied.
EDIT
It's difficult to be certain without more information about the database schema, but at a guess the issue is with this statement:
new SqlBulkCopyColumnMapping(c.ColumnName, c.ColumnName)
From your description, it sounds like not all the columns in the source table exist in the destination table. You need a filter in your SqlBulkCopyColumnMapping construction loop to skip any columns which do not exist in the destination.
My C# is not good enough to give a example which I'm confident will work, but in pseudocode it would be
foreach column c in sourcetable
{
if c.ColumnName exists in destination_table.columns
{
new SqlBulkCopyColumnMapping(c.ColumnName, c.ColumnName)
}
}
(I'm sure it's possible to convert this to a lambda expression)
Note that this is not particularly robust in the scenario where the column names match but the datatypes are incompatible.

Ed Harper, this is what it looks like without pseudo code
(in this case from DataTable dt (fully defined) to an existing table in the db:
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString))
{
bulkCopy.DestinationTableName = "dbo.DepartmentsItems";
// Write from the source to the destination.
foreach (DataColumn c in dt.Columns)
{
bulkCopy.ColumnMappings.Add(c.ColumnName, c.ColumnName);
}
bulkCopy.WriteToServer(dt);
return dt.Rows.Count;
}

try this:SqlBulkCopyColumnMapping Class
Hope you are looking for the same