I'm trying to create code that adds column 'ExternalId' to almost every table in foreach loop with some exceptions. If table schema name equals "Scheduler" or table name starts with "users" dont add column.
First this I probably need is a list of schemas and tables.
public class AddExternalIdColumnsToManyTables : Migration
{
public override void Up()
{
// schema name example
// Scheduler.Job
// dbo.users_logins
var schemaAndTableNames = new List<string>();
foreach (var item in schemaAndTableNames)
{
if (item.StartsWith("Scheduler"))
{
continue;
}
if (item.Split('.')[1].StartsWith("users"))
{
continue;
}
Alter.Table(item.Split('.')[1]).AddColumn("ExternalId").AsInt32().Nullable();
}
If you're asking how to get table names:
MigrationBase class (Migration's parent) has ConnectionsString property. You can easily get it and write ADO.net query to get table names. Simple example:
var builder = new SqlConnectionStringBuilder(ConnectionString);
var query = "SELECT TABLE_NAME" +
"FROM INFORMATION_SCHEMA.TABLES" +
"WHERE TABLE_TYPE = 'BASE TABLE' AND " +
$"TABLE_CATALOG = '{builder.InitialCatalog}'"
using (var connection = new SqlConnection(builder.ConnectionString))
{
connection.Open();
var command = new SqlCommand(query, connection);
using (var sqlReader = command.ExecuteReader(CommandBehavior.KeyInfo))
{
var schemaTable = sqlReader.GetSchemaTable();
foreach (DataRow row in schemaTable.Rows)
{
///your table names
}
}
connection.Close();
}
Related
This questions had been posted several times (1,2,3,4), but I did not find one that would apply to my case.
I have a Dictionary with the structure:
public Dictionary<Int32, PhaseMag> Data;
Where PhaseMag:
public struct PhaseMag
{
public Single Magnitude;
public Single Phase;
}
Each Key value will contain two 2 values (Mag. and Phase). If you prefer to see an image.
I need to store all the content of this Dictionary in a unique Table in the DB (Microsoft SQL). Each line of the Dictionary should become one line of my DB Table. My final table will contain 3 fields, 1) Key 2) Mag and 3) Phase. For example, if I have 30 Keys, my table will contain 30 lines, one for each key.
My ideas:
Create a foreach loop based on each Key and create an insert into to the DB (one for Key->Mag and other for Key->Phase). But I don't think that this will be the best approach, especially, because my dictionary contains several thousand of lines.
So, what should be my approach to do this? I simply need to save my Dictionary into the DB, which each line of the Dic. will be one line from the DB.
Assume you're using SQL Server, here're two of many options you have:
Bulk insert, it's recommended.
public void BulkWrite(Dictionary<Int32, PhaseMag> data)
{
var dataTable = new DataTable();
dataTable.Columns.Add(new DataColumn { DataType = typeof(int), ColumnName = "Key" });
dataTable.Columns.Add(new DataColumn { DataType = typeof(Single), ColumnName = "Magnitude" });
dataTable.Columns.Add(new DataColumn { DataType = typeof(Single), ColumnName = "Phase" });
foreach (var x in data)
{
var r = dataTable.NewRow();
dataTable.Rows.Add(r);
r[0] = x.Key;
r[1] = x.Value.Magnitude;
r[2] = x.Value.Phase;
}
using (var conn = new SqlConnection("conneciton string"))
{
conn.Open();
using (var bulkCopy = new SqlBulkCopy(conn))
{
bulkCopy.BatchSize = 4000;
bulkCopy.DestinationTableName = "YorTableName";
bulkCopy.WriteToServer(dataTable);
}
}
}
Multiple inline query, executed as batch. As all your data are number, so low risk of SQL injection even using inline query.
public void InlineQueryWrite(Dictionary<Int32, PhaseMag> data)
{
using (var conn = new SqlConnection("conneciton string"))
{
conn.Open();
foreach (var bulk in data.Select((d, i) => new {d, i}).GroupBy(x => x.i % 10))
{
var sb = new StringBuilder();
foreach (var x in bulk)
{
sb.AppendFormat("Insert Into Your_Table (Key, Magnitude, Phase) Values ({0},{1},{2});", x.d.Key, x.d.Value.Magnitude, x.d.Value.Phase);
}
using (var command = conn.CreateCommand())
{
command.CommandText = sb.ToString();
command.ExecuteNonQuery();
}
}
}
}
I haven't ran/tested through the code, but they should work.
I am trying to bulk copy from one table to another by mapping the column names as the source and destination may not have same columns always.
Source can have 8 columns and destination can have 10 .. I need to map the columns and bulk copy.
Tried the below code..didn't work..getting Error: The given ColumnName 'moduleid' does not match up with any column in data source.
Source: existingtablecolumnsPresent has [collection time],[logtime],[moduleid],[node],[reason],[time],[timestamp],[usecaseid]
Destination: dataTable.Columns has [Node],[Time],[Reason],[Moduleid],[Usecaseid]
Please advise
public static void BatchBulkCopy(DataTable dataTable, string DestinationTbl, List<string> columnMapping,string filename)
{
var program = new Program();
// Get the DataTable
DataTable dtInsertRows = dataTable;
using (SqlBulkCopy sbc = new SqlBulkCopy(program.connectionStr.ToString()))
{
try {
sbc.DestinationTableName = DestinationTbl.ToLower();
string sourceTableQuery = "Select top 1 * from " + "[" + dataTable.TableName + "]";
DataTable dtSource = SqlHelper.ExecuteDataset(program.connectionStr.ToString(), CommandType.Text, sourceTableQuery).Tables[0];
for (int i = 0; i < dataTable.Columns.Count; i++)
{ //check if destination Column Exists in Source table
if (dtSource.Columns.Cast<DataColumn>().Select(a => "[" + a.ColumnName.ToLower() + "]").Contains(dataTable.Columns[i].ToString().ToLower()))//contain method is not case sensitive
{
List<string> existingtablecolumnsPresent = dtSource.Columns.Cast<DataColumn>().Select(a => "[" + a.ColumnName.ToLower() + "]").Distinct().OrderBy(t => t).ToList();
int sourceColumnIndex = existingtablecolumnsPresent.IndexOf(dataTable.Columns[i].ToString().ToLower());//Once column matched get its index
sbc.ColumnMappings.Add(dtSource.Columns[sourceColumnIndex].ToString(), dtSource.Columns[sourceColumnIndex].ToString());//give coluns name of source table rather then destination table so that it would avoid case sensitivity
}
}
sbc.WriteToServer(dtInsertRows);
sbc.Close();
}
catch (Exception ex)
{
Log.WriteLog("BatchBulkCopy" + " - " + filename, dataTable.TableName, ex.Message.ToString());
// To move a file or folder to a new location:
//if (File.Exists(program.sourceFile + filename))
// System.IO.File.Move(program.sourceFile + filename, program.failedfiles + filename);
}
As requested (create a DataTable with the columns you want to insert in them- leave the others out. Make sure any columns you leave out are marked in the table for NULL or have a DEFAULT VALUE constraint (I can't show you how to do that unless you show me your table);
//This first method is psuedoCode to explain how to create your datatable. You need to do it in the way that makes sense for you.
public DataTable createDataTable(){
List<string> excludedColumns = new List<string>();
excludedColumns.Add("FieldToExclude");
//...
DataTable dt = new DataTable();
foreach(string col in getColumns(myTable)){
if(!excludedColumns.Contains(name)){
DataColumn dC = new DataColumn(name,type);
DataTable.Add(dC);
}
return dt;
}
public List<string> getColumns(string tableName)
{
List<string> ret = new List<string>();
using (SqlConnection conn = getConn())
{
conn.Open();
using (SqlCommand com = conn.CreateCommand())
{
com.CommandText = "select column_Name from information_schema.COLUMNS where table_name = #tab";
com.Parameters.AddWithValue("#tab", tableName);
SqlDataReader read = com.ExecuteReader();
While(read.Read()){
ret.Add(Convert.ToString(read[0]);
}
conn.Close();
}
return ret;
}
//Now, you have a DataTable that has all the columns you want to insert. Map them yourself in code by adding to the appropriate column in your datatable.
public bool isCopyInProgess = false;//not necessary - just part of my code
public void saveDataTable(string tableName, DataTable table)
{
using (SqlConnection conn = getConn())
{
conn.Open();
using (var bulkCopy = new SqlBulkCopy(conn))//, SqlBulkCopyOptions.KeepIdentity))//un-comment if you want to use your own identity column
{
// my DataTable column names match my SQL Column names, so I simply made this loop. However if your column names don't match, just pass in which datatable name matches the SQL column name in Column Mappings
foreach (DataColumn col in table.Columns)
{
//Console.WriteLine("mapping " + col.ColumnName+" ("+does_Column_Exist(col.ColumnName,"Item").ToString()+")");
bulkCopy.ColumnMappings.Add(col.ColumnName, "["+col.ColumnName+"]");
// Console.WriteLine("ok\n");
}
bulkCopy.BulkCopyTimeout = 8000;
bulkCopy.DestinationTableName = tableName;
bulkCopy.BatchSize = 10000;
bulkCopy.EnableStreaming = true;
//bulkCopy.SqlRowsCopied += BulkCopy_SqlRowsCopied;
//bulkCopy.NotifyAfter = 10000;
isCopyInProgess = true;
bulkCopy.WriteToServer(table);
}
conn.Close();
}
}
Also, use this as your bolumn checker:
public bool does_Column_Exist(string colName,string tableName)
{
bool ret = false;
using (SqlConnection conn = getConn())
{
conn.Open();
using (SqlCommand com = conn.CreateCommand())
{
com.CommandText = "select count(*) from information_schema.COLUMNS where column_name = #col and table_name = #tab";
com.Parameters.AddWithValue("#tab", tableName);
com.Parameters.AddWithValue("#col", colName);
ret = Convert.ToInt32(com.ExecuteScalar()) == 0 ? false : true;
}
conn.Close();
}
return ret;
}
Is there a specific reason you need C# for this? It seems like the path of least resistance would be to use SQL to do the job.
INSERT INTO table2
(column_name(s))
SELECT column_name(s)
FROM table1;
Not sure if this is written correctly but it looks correct. I am wanting to update a record if the id already exists and insert if not.
DataSet ds = new DataSet();
ds.ReadXml(XDocument.Load(Application.StartupPath + #"\xml1.xml").CreateReader());
using (var conn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0; Data Source=" + Application.StartupPath + "\\Database3.mdb"))
{
conn.Open();
// make two commands here
var commInsert = new OleDbCommand("Insert INTO Table1 (description, active) VALUES (#iq_question,#active);", conn);
var commUpdate = new OleDbCommand("UPDATE Table1 SET description=#iq_question,active=#active WHERE ID=#question_id;", conn);
// here add your parameters with no value
//string question_id = row[0].ToString();
//string iq_question = row[1].ToString();
//string active = row[4].ToString();
commInsert.Parameters.Add(new OleDbParameter("#iq_question", OleDbType.VarChar));
commInsert.Parameters.Add(new OleDbParameter("#active", OleDbType.VarChar));
commUpdate.Parameters.Add(new OleDbParameter("#question_id", OleDbType.AutoNumber));
commUpdate.Parameters.Add(new OleDbParameter("#iq_question", OleDbType.Text));
commUpdate.Parameters.Add(new OleDbParameter("#active", OleDbType.Text));
foreach (DataTable table in ds.Tables)
{
foreach (DataRow row in table.Rows)
{
// here only reset the values
commUpdate.Parameters["#question_id"].Value = row[0].ToString();
commUpdate.Parameters["#iq_question"].Value = row[1].ToString();
commUpdate.Parameters["#active"].Value = row[4].ToString();
int recs = commUpdate.ExecuteNonQuery();
if (recs < 1) // when no records updated do insert
{
commInsert.Parameters["#iq_question"].Value = row[1].ToString();
commInsert.Parameters["#active"].Value = row[4].ToString();
commInsert.ExecuteNonQuery();
}
}
}
commInsert.Dispose();
commUpdate.Dispose();
conn.Close();
}
System.Windows.Forms.MessageBox.Show("Updated Latest Data Was Succesfull");
I either get an error on the insert saying it will create duplicate content, or it creates more rows with different data. So say I should be getting 10 rows from the xml file, the first time I run it I get the 10 rows with the correct data. If I run it again, I end up with 10 more so being 20 but the last 10 rows show different data. I don't think I am identifying the rows in the xml file correctly and I need to do some research on that part.
There is no Exists for MS Access. The engine is much more primitive than Sql Server. See here: Microsoft Access SQL. I think, what you can do is:
myCommand.CommandText = "UPDATE Table1 SET description=#iq_question,active=#active WHERE ID=#currentRow";
......
int recs = myCommand.ExecuteNonQuery();
if (recs < 1) // when no records updated do insert
{
myCommand.Parameters.Clear();
myCommand.CommandText = "Insert INTO Table1 VALUES(#iq_question,#active)";
.....
}
This is still 2 statements but you can save some coding by not doing Select first. Because ExecuteNonQuery will tell you if you updated anything
Another thing is that your code is a bit inefficient. You have nested loop where you can reuse same command and connection. Yuu can do this
using (var conn = new OleDbConnection(.......))
{
conn.Open();
// make two commands here
var commInsert = new OleDbCommand(.....);
var commUpdate = new OleDbCommand(.....);
// here add your parameters with no value
commInsert.Parameters.Add(new OleDbParameter(....));
.......
Foreach (....)
{
Foreach (....)
{
// here only reset the values
commUpdate.Parameters[0].Value = ...
...
int recs = commUpdate.ExecuteNonQuery();
if (recs < 1) // when no records updated do insert
{
commInsert.Parameters[0].Value = iq_question;
.....
}
}
}
commInsert.Dispose();
commUpdate.Dispose();
}
You can also use nested using for commands. Only setting values will be more efficient to what you do right now.
I'm trying to get the column names of a table I have stored in SQL Server 2008 R2.
I've literally tried everything but I can't seem to find how to do this.
Right now this is my code in C#
public string[] getColumnsName()
{
List<string> listacolumnas=new List<string>();
using (SqlConnection connection = new SqlConnection(Connection))
using (SqlCommand command = connection.CreateCommand())
{
command.CommandText = "SELECT TOP 0 * FROM Usuarios";
connection.Open();
using (var reader = command.ExecuteReader(CommandBehavior.KeyInfo))
{
reader.Read();
var table = reader.GetSchemaTable();
foreach (DataColumn column in table.Columns)
{
listacolumnas.Add(column.ColumnName);
}
}
}
return listacolumnas.ToArray();
}
But this is returning me the following
<string>ColumnName</string>
<string>ColumnOrdinal</string>
<string>ColumnSize</string>
<string>NumericPrecision</string>
<string>NumericScale</string>
<string>IsUnique</string>
<string>IsKey</string>
<string>BaseServerName</string>
<string>BaseCatalogName</string>
<string>BaseColumnName</string>
<string>BaseSchemaName</string>
<string>BaseTableName</string>
<string>DataType</string>
<string>AllowDBNull</string>
<string>ProviderType</string>
<string>IsAliased</string>
<string>IsExpression</string>
<string>IsIdentity</string>
<string>IsAutoIncrement</string>
<string>IsRowVersion</string>
<string>IsHidden</string>
<string>IsLong</string>
<string>IsReadOnly</string>
<string>ProviderSpecificDataType</string>
<string>DataTypeName</string>
<string>XmlSchemaCollectionDatabase</string>
<string>XmlSchemaCollectionOwningSchema</string>
<string>XmlSchemaCollectionName</string>
<string>UdtAssemblyQualifiedName</string>
<string>NonVersionedProviderType</string>
<string>IsColumnSet</string>
Any ideas?
It shows the <string> tags as this is how my web service sends the data.
You can use the query below to get the column names for your table. The query below gets all the columns for a user table of a given name:
select c.name from sys.columns c
inner join sys.tables t
on t.object_id = c.object_id
and t.name = 'Usuarios' and t.type = 'U'
In your code, it will look like that:
public string[] getColumnsName()
{
List<string> listacolumnas=new List<string>();
using (SqlConnection connection = new SqlConnection(Connection))
using (SqlCommand command = connection.CreateCommand())
{
command.CommandText = "select c.name from sys.columns c inner join sys.tables t on t.object_id = c.object_id and t.name = 'Usuarios' and t.type = 'U'";
connection.Open();
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
listacolumnas.Add(reader.GetString(0));
}
}
}
return listacolumnas.ToArray();
}
I typically use the GetSchema method to retrieve Column specific information, this snippet will return the column names in a string List:
using (SqlConnection conn = new SqlConnection("<ConnectionString>"))
{
string[] restrictions = new string[4] { null, null, "<TableName>", null };
conn.Open();
var columnList = conn.GetSchema("Columns", restrictions).AsEnumerable().Select(s => s.Field<String>("Column_Name")).ToList();
}
SELECT COLUMN_NAME
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = 'YourTable'
public string[] getColumnsName()
{
List<string> listacolumnas=new List<string>();
using (SqlConnection connection = new SqlConnection(Connection))
using (SqlCommand command = connection.CreateCommand())
{
command.CommandText = "select column_name from information_schema.columns where table_name = 'Usuarios'";
connection.Open(;
using (var reader = command.ExecuteReader(CommandBehavior.KeyInfo))
{
reader.Read();
var table = reader.GetSchemaTable();
foreach (DataColumn column in table.Columns)
{
listacolumnas.Add(column.ColumnName);
}
}
}
return listacolumnas.ToArray();
}
The original post was close to the goal, Just some small changes and you got it.
Here is my solution.
public List<string> GetColumns(string tableName)
{
List<string> colList = new List<string>();
DataTable dataTable = new DataTable();
string cmdString = String.Format("SELECT TOP 0 * FROM {0}", tableName);
if (ConnectionManager != null)
{
try
{
using (SqlDataAdapter dataContent = new SqlDataAdapter(cmdString, ConnectionManager.ConnectionToSQL))
{
dataContent.Fill(dataTable);
foreach (DataColumn col in dataTable.Columns)
{
colList.Add(col.ColumnName);
}
}
}
catch (Exception ex)
{
InternalError = ex.Message;
}
}
return colList;
}
Currently, there are two ways I could think of doing this:
In pure SQL Server SQL you can use the views defined in INFORMATION_SCHEMA.COLUMNS. There, you would need to select the row for your table, matching on the column TABLE_NAME.
Since you are using C#, it's probably easier to obtain the names from the SqlDataReader instance that is returned by ExecuteReader. The class provides a property FieldCount, for the number of columns, and a method GetName(int), taking the column number as its argument and returning the name of the column.
sp_columns - http://technet.microsoft.com/en-us/library/ms176077.aspx
There are many built in stored procedures for this type of thing.
Lets say I have a table in SQLServer named MyTable
ID FirstName LastName
1 Harry Dan
2 Maria Vicente
3 Joe Martin
Now if I have to insert any data in table, I will simply fire Insert Query like this
INSERT INTO MyTable (ID, FirstName, LastName) VALUES (4, Smith, Dan);
But what if I don't know the column names beforehand, I only know table name. Then is there a way to get the column name of table at runtime?
You can use sql-
SELECT name FROM sys.columns WHERE object_id = OBJECT_ID('TABLE_NAME')
Or you can query for SELECT TOP 0 * FROM TableName. Then, you can get the columns:
using(var reader = command.ExecuteReader())
{
reader.Read();
var table = reader.GetSchemaTable();
foreach (DataColumn column in table.Columns)
{
Console.WriteLine(column.ColumnName);
}
}
Another option using pure C# / .NET code:
First a helper method, that here returns a simple list of column names. Using a DataTable to hold table schema information, means that other information can also be retreived for each column, fx. if it is an AutoIncreament column etc.
private IEnumerable<string> GetColumnNames(string conStr, string tableName)
{
var result = new List<string>();
using (var sqlCon = new SqlConnection(conStr))
{
sqlCon.Open();
var sqlCmd = sqlCon.CreateCommand();
sqlCmd.CommandText = "select * from " + tableName + " where 1=0"; // No data wanted, only schema
sqlCmd.CommandType = CommandType.Text;
var sqlDR = sqlCmd.ExecuteReader();
var dataTable = sqlDR.GetSchemaTable();
foreach (DataRow row in dataTable.Rows) result.Add(row.Field<string>("ColumnName"));
}
return result;
}
The method can be called as:
var sortedNames = GetColumnNames("Data Source=localhost;Initial Catalog=OF2E;Integrated Security=SSPI", "Articles").OrderBy(x => x);
foreach (var columnName in sortedNames) Console.WriteLine(columnName);
Simply by this Query :
SELECT * FROM sys.columns
WHERE object_id = OBJECT_ID('dbo.[table_name]')
OR This Query :
SELECT COLUMN_NAME
FROM information_schema.columns
WHERE TABLE_NAME = [table_name]
ORDER BY COLUMN_NAME
var ColName = "";
var model = new AttributeMappingSource().GetModel(typeof(DataClassesDataContext));
foreach (var mt in model.GetTables())
{
if (mt.TableName == "dbo.Table_Name")
{
foreach (var dm in mt.RowType.DataMembers)
{
ColName = dm.MappedName + ", ";
Response.Write(ColName);
}
}
}
This question was answered before in various threads
check:
How can I get column names from a table in SQL Server?
How can I get column names from a table in Oracle?
How do I list all the columns in a table?