I have two DB tables, they have the same columns but their data types are different(E.g.: "Check" column is of a type integer in table 1, but varchar in table2). I am trying to copy the data from one table to another by using BulkCopy. I have a code like:
using (SqlBulkCopy sbc = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity))
{
cmdSQLT = new SqlCommand("SELECT " + ColumnsNames + " FROM [transfer].[" + SelectedScheme + ".OldTable]", conn);
cmdSQLT.CommandTimeout = 1200;
reader = cmdSQLT.ExecuteReader();
sbc.ColumnMappings.Add("CHECK", "CHECK");
sbc.DestinationTableName = "[" + SelectedScheme + "_Newtable]";
sbc.BulkCopyTimeout = 1200;
sbc.WriteToServer(reader);
}
I am getting an error saying
The locale id '0' of the source column 'CHECK' and the locale id
'1033' of the destination column 'CHECK' do not match.
This is happening due to the data types differences between the tables. How can I make data type conversion in the previous code?
Your help is much appreciated!
You can do the conversion on source select using a CAST() statement.
However, if the target connection have access to the source database then instead of doing a SqlBulkCopy a single "insert into < target > select ... from < source >" statement would be a much more effective solution.
ie:
var colNames = ColumnsNames.Split(',').ToArray();
for(int i=0;i< colNames.Length;i++)
{
if (colNames[i].ToUpper() == "CHECK")
{
colNames[i] = "cast(Check as varchar(10))"
}
}
ColumnsNames = string.Join(",",colNames);
Might not be what you expect but for simply updating one table based on the rows of another table, improved performance and scalability can be achieved with basic INSERT, UPDATE, and DELETE statements. For example:
INSERT tbl_A (col, col2)
SELECT col, col2
FROM tbl_B
WHERE NOT EXISTS (SELECT col FROM tbl_A A2 WHERE A2.col = tbl_B.col);
If it's more about column/table sync, the Merge keyword could be what you are looking for.
Related
How can I pass a columns name by parameter,
follow an example
DataTable dt = new DataTable();
// Here I fill my datatable
for (int i = 0; i < dt.Rows.Count; i++)
{
for (int j = 0; j < dt.Columns.Count; j++)
{
string columnsname = dt.Rows[i][dt.columns[j].toString()].toString();
SqlCommand comando = new SqlCommand();
comando.commandText = "UPDATE Sales.Store SET #columnname = #demographics where id = #id";
comando.Parameters.Add(new SqlParameter("#columnname", columname));
comando.Parameters.Add(new SqlParameter("#dados2", dados2));
comando.ExecuteNonQuery();
comando.Clear();
comando.Dispose()
}
}
This doesn't work, but I have 88 columns, and I need update all data in every 88 columns in each row.
You cannot parameterize column names.
To do what you want you will need to resort to dynamic SQL.
Well, if you have 30,000 rows with 88 columns, and you need to update all 88 columns, you probably want to rethink your database schema.
Itay.
I have figured out a way to include a work around for parametrized column names. I had the same problem but came up with a different way and since I would be the only one using the column names then I believe this is still a safe bet.
String sqlcomm = "SELECT * FROM Asset WHERE " + assetColName + " = ";
command.CommandText = sqlcomm + "$assetColValue";
//command.CommandText = #"SELECT * FROM Asset WHERE $assetColName = '$assetColValue'";
//command.Parameters.AddWithValue("$assetColName", assetColName);
command.Parameters.AddWithValue("$assetColValue", assetColValue);
As you can see from the code above. I tried almost what you did which I then had to comment out. I then concatenated strings together and was able to use my parametrized column name and value which then the value is securely added. The column name however is not secured but this is a method that only I will be using so its still somewhat safe. You can add regular expressions if you want to be more secure but you get the idea of the fix.
Just concatenate the sql string:
"UPDATE Contracts set " + columnName + " = #columnValue where ID = #ID"
Where column name is a string that represents a column in the table
I have a datatable that may have 1000 or so rows in it. I need to go thru the datatable row by row, get the value of a column, run a query (Access 2007 DB) and update the datatable with the result. Here's what I have so far, which works:
String FilePath = "c:\\MyDB.accdb";
string QueryString = "SELECT MDDB.NDC, MDDB.NDC_DESC "
+ "FROM MDDB_MASTER AS MDDB WHERE MDDB.NDC = #NDC";
OleDbConnection strAccessConn = new OleDbConnection(string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + FilePath));
strAccessConn.Open();
OleDbDataReader reader = null;
int rowcount = InputTable.Rows.Count; //InputTable is the datatable
int count = 0;
while (count < rowcount)
{
string NDC = InputTable.Rows[count]["NDC"].ToString();
//NDC is a column in InputTable
OleDbCommand cmd = new OleDbCommand(QueryString, strAccessConn);
cmd.Parameters.Add("#NDC", OleDbType.VarChar).Value = NDC;
reader = cmd.ExecuteReader();
while (reader.Read())
{
//update the NDCDESC column with the query result
//the query should only return 1 line
dataSet1.Tables["InputTable"].Rows[count]["NDCDESC"] = reader.GetValue(1).ToString();
}
dataGridView1.Refresh();
count++;
}
strAccessConn.Close();
However this seems very inefficient since the query needs to run one time for each row in the datatable. Is there a better way?
You're thinking of an update query. You don't actually have to go over every row one by one. SQL is a set based language, so you only have to write a single statement that it should do for all rows.
Do this:
1) Create > Query Design
2) Close the dialog that selects tables
3) Make sure you're in sql mode (top left corner)
4) Paste this:
UPDATE INPUTTABLE
INNER JOIN MDDB_MASTER ON INPUTTABLE.NDC = MDDB_MASTER.NDC
SET INPUTTABLE.NDCDESC = [MDDB_MASTER].[NDC_DESC];
5) Switch to design mode to see what happens. You may have to correct Input table, I couldn't find it's name. I'm assuming they;re both in the same database.
You'll see the query type is now an update query.
You can run this text through cmd.ExecuteNonQuery(sql) and the whole thing should run very quickly. If it doesn't you'll need an index on one of the tables;
THis works by joining the two table on NDC and then copying the NDC_DESC over from MDDB_MASTER to the inputtable.
I missed the part about InputTable coming from Excel.
For better speed, instead of executing the query in Access over and over, you can get all rows from MDDB_MASTER into a datatable in one select statement:
SELECT MDDB.NDC, MDDB.NDC_DESC FROM MDDB_MASTER
And then use the DataTable.Select method to filter the right row.
mddb_master.Select("NDC = '" + NDC +'")
This will be done in memory and should be much faster than all the round trips you have now. Especially over the network these round trips are expensive. 225k rows should be only a few MB (roughly a JPEG image) so that shouldn't be an issue.
You could use the "IN" clause to build a bigger query such as:
string QueryString = "SELECT MDDB.NDC, MDDB.NDC_DESC "
+ "FROM MDDB_MASTER AS MDDB WHERE MDDB.NDC IN (";
int rowcount = InputTable.Rows.Count; //InputTable is the datatable
int count = 0;
while (count < rowcount)
{
string NDC = InputTable.Rows[count]["NDC"].ToString();
QueryString += (count == 0 ? "" : ",") + "'" + NDC + "'";
}
QueryString += ")";
You can optimize that with StringBuilders since that could be a lot of strings but that's a job for you. :)
Then in a single query, you would get all the NDC descriptions you need and avoid performing 1000 queries. You would then roll through the reader, find values in the InputTable, and update them. Of course, in this case, you're looping through the InputTable multiple times but it might be a better option. Especially if yor InputTable could hold duplicate NDC values.
Also, note that you have a OleDbDataReader leak in your code. You keep reassigning the reader reference to a new instance of a reader before disposing of the old reader. Same with commands. You keep instantiating a new command but are not disposing of it properly.
I am developing an application in C# in Visual Studio 2008. I connected a SQL Server 2008 database with it.
I want to count the number of columns so that I can loop around them to get the particular data.
I can figure it out columns by going to the database but I am joing 4-5 tables in my programs so I want to know if I can count the columns.
Can anyone help me in this?
Thank you
Shyam
select count(*) from INFORMATION_SCHEMA.columns where TABLE_NAME = 'YourTableName'
Something like this ?
SELECT COUNT(*)
FROM sys.columns c
INNER JOIN sys.tables t ON c.object_id = t.object_id
WHERE t.name = 'yourTable'
See this page provided wy TaronPro to know how to retrieve the result.
If you are using SQLConnection object to connect to DB, use its GetSchema method to get list of all columns without querying.
using (SqlConnection connection = new SqlConnection(connectionString))
{
// Connect to the database then retrieve the schema information.
connection.Open();
DataTable table = connection.GetSchema("Tables");
..
..
..
If you want to know columns for specific owner, table or table type, use restriction within GetSchema method.
string[] restrictions = new string[4];
restrictions[1] = "dbo";
DataTable table = connection.GetSchema("Tables", restrictions);
for more information refer this link.
What I did in a similar situation is that when I executed the query I retrieved all the data into a DataSet.
When I got the DataSet I opened the first table (ds.Tables[0]). Obviously you check first for existance.
When you have the table then its as simple as performing a
dt.Columns.Count;
In summary DS.Tables[0].Columns.Count
To find a specific column by name you loop through and find the one that
for (z=0; z < dt.Columns.Count; z++)
{
// check to see if the column name is the required name passed in.
if (dt.Columns[z].ColumnName == fieldName)
{
// If the column was found then retrieve it
//dc = dt.Columns[z];
// and stop looking the rest of the columns
requiredColumn = z;
break;
}
}
Then to find the data you need I would then loop through the rows of the table and get the field for that column...
ie...
string return = dr.Field<string>(requiredColumn);
Probalby not the best way of doing it but it works. Obviously if the data contained in the field is not string then you need to pass the appropriate type...
dr.Field<decimal>(requiredColumn)
dr.Field<int>(requiredColumn)
etc
Rgds
George
The reader itself gives you the number of columns. This is useful when you don't want to know the number rows of a specific table or view but from an ad-hoc query.
You can dump the columns like this
string sql = "SELECT * FROM my query";
SqlCommand cmd = new SqlCommand(sql, connection);
using (SqlDataReader reader = cmd.ExecuteReader()) {
while (reader.Read()) {
for (int i = 0; i < reader.FieldCount; i++) {
Console.WriteLine("{0} = {1}",
reader.GetName(i),
reader.IsDBNull(i) ? "NULL" : reader.GetValue(i));
}
Console.WriteLine("---------------");
}
}
you can use Microsoft.SqlServer.Management.Smo namespace to get the number of columns in a specified table as follows
1 . add Microsoft.SqlServer.Management.Smo dll in your project and use the namespace Microsoft.SqlServer.Management.Smo
2 . write the follwing code
private int colCount()
{
Server server=new Server(".\\SQLEXPRESS");
Database database=Server.Databases["your database name"];
Table table=database.Tables["your table name"];
return (table.Columns.Count);
}
How can I pass a columns name by parameter,
follow an example
DataTable dt = new DataTable();
// Here I fill my datatable
for (int i = 0; i < dt.Rows.Count; i++)
{
for (int j = 0; j < dt.Columns.Count; j++)
{
string columnsname = dt.Rows[i][dt.columns[j].toString()].toString();
SqlCommand comando = new SqlCommand();
comando.commandText = "UPDATE Sales.Store SET #columnname = #demographics where id = #id";
comando.Parameters.Add(new SqlParameter("#columnname", columname));
comando.Parameters.Add(new SqlParameter("#dados2", dados2));
comando.ExecuteNonQuery();
comando.Clear();
comando.Dispose()
}
}
This doesn't work, but I have 88 columns, and I need update all data in every 88 columns in each row.
You cannot parameterize column names.
To do what you want you will need to resort to dynamic SQL.
Well, if you have 30,000 rows with 88 columns, and you need to update all 88 columns, you probably want to rethink your database schema.
Itay.
I have figured out a way to include a work around for parametrized column names. I had the same problem but came up with a different way and since I would be the only one using the column names then I believe this is still a safe bet.
String sqlcomm = "SELECT * FROM Asset WHERE " + assetColName + " = ";
command.CommandText = sqlcomm + "$assetColValue";
//command.CommandText = #"SELECT * FROM Asset WHERE $assetColName = '$assetColValue'";
//command.Parameters.AddWithValue("$assetColName", assetColName);
command.Parameters.AddWithValue("$assetColValue", assetColValue);
As you can see from the code above. I tried almost what you did which I then had to comment out. I then concatenated strings together and was able to use my parametrized column name and value which then the value is securely added. The column name however is not secured but this is a method that only I will be using so its still somewhat safe. You can add regular expressions if you want to be more secure but you get the idea of the fix.
Just concatenate the sql string:
"UPDATE Contracts set " + columnName + " = #columnValue where ID = #ID"
Where column name is a string that represents a column in the table
I'm wanting to get a list of the column names returned from a SQL SELECT statement. Can someone suggest an easy way to do this?
I have a tool that lets users define a query using any SQL SELECT statement. The results of the query are then presented in a custom manner. To set up the presentation, I need to know the column names so that the user can store formatting settings about each column.
Btw, the formatting settings are all being created via ASP.NET web pages, so the query results will end up in .NET if that helps with any ideas people have.
Any ideas?
You should be able to do this using the GetName method. Something like this probably:
SqlDataReader mySDR = cmd.ExecuteReader();
for(int i = 0;i < mySDR.FieldCount; i++)
{
Console.WriteLine(mySDR.GetName(i));
}
This is something you could do entirely from a asp.net page. No special/extra SQL required.
Assuming SQL Server: You could use SET FMTONLY to just return metadata (and not the actual data), e.g.:
USE AdventureWorks2008R2;
GO
SET FMTONLY ON;
GO
SELECT *
FROM HumanResources.Employee;
GO
SET FMTONLY OFF;
GO
You can get by something as following
Note : You need to fill the DataTable of the Dataset.........
DataSet1 DataSet1 = new DataSet1();
DataTable dt = DataSet1.Tables(0);
DataColumn dc = null;
foreach (DataColumn dc_loopVariable in dt.Columns) {
dc = dc_loopVariable;
Response.write(dc.ColumnName.ToString() + " " + dc.DataType.ToString() + "<br>");
}
Another method to just return meta data is
select top 0 * from table
If you know the table name you could try using:
desc <table_name>
I'm assuming you are using SQL Server.
Or as an alternative:
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'TableNameGoesHere'
ORDER BY ORDINAL_POSITION
You might want to use the second option if you are going to be using ASP.NET
This will get you more than the column name if you need more information about each column like size, ordinal,etc. A few of the most important properties are listed, but there are more.
Note, DataObjects.Column is a POCO for storing column information. You can roll your own in your code. Also, note I derive the .Net type as well, useful for converting SQL data types to .Net (C#) ones. ConnectionString and TableName would be supplied from a caller.
using (SqlConnection conn = new SqlConnection(ConnectionString))
{
conn.Open();
SqlCommand comm = new SqlCommand("Select top(1) * from " + TableName + " Where 1=0");
comm.CommandType = CommandType.Text;
comm.Connection = conn;
using (SqlDataReader reader = comm.ExecuteReader(CommandBehavior.KeyInfo))
{
DataTable dt = reader.GetSchemaTable();
foreach (DataRow row in dt.Rows)
{
//Create a column
DataObjects.Column column = new DataObjects.Column();
column.ColumnName = (string)row["ColumnName"];
column.ColumnOrdinal = (int)row["ColumnOrdinal"];
column.ColumnSize = (int)row["ColumnSize"];
column.IsIdentity = (bool)row["IsIdentity"];
column.IsUnique = (bool)row["IsUnique"];
//Get the C# type of data
object obj = row["DataType"];
Type runtimeType = obj.GetType();
System.Reflection.PropertyInfo propInfo = runtimeType.GetProperty("UnderlyingSystemType");
column.type = (Type)propInfo.GetValue(obj, null);
//Set a string so we can serialize properly later on
column.DataTypeFullName = column.type.FullName;
//I believe this is SQL Server Data Type
column.SQLServerDataTypeName = (string)row["DataTypeName"];
//Do something with the column
}
}
}