I want to perform insert into database but in different Tables. Each record i have in my file is assigned with the table Identifiers which will be used as a key. For now what i have done is to make it a generic approach, I have created one more file.txt in which i have specified all the identifiers with the table names. and stored it into a dictionary of string, string.
So, I have example some thing like below in my dictionary:
Table Identifier, Table Name
Table Identifier, Table Name
Then I created another dictionary of string as key (Note: I have used the previous dictionary value as a key into this dictionary) and list of string as values to get the column names of the table name.
So, No I have some thing like this below sample data into my dictionary:
Table Name, Column Names IEnumerable
Table Name, Column Names IEnumerable
Then,
The datafile.txt which contains the data as pipe delimited, I split them and saved into a List of KVP of string and List of String. As I mentioned before, I have table identifiers in my data files. So i used them as key in my KVP and store the split values into List of string.
So, No I have some thing like this below sample data into my List of KVP of string , List of string:
Table Identifier, IEnumerable Values
Table Identifier, IEnumerable Values
Now doing half of the work, I am stuck into my FINAL ISSUE:
Now I have All the identifers , Table Names, Column Names and Values with me into the dictionary and List. And only thing to do is to match and merge the records and DUMP it!
For matching: I have thought to match the List of KVPs key with the dictionarys key and then use the values as a key to get the Column Names.
Expected Scenario Image:
List [[List< Column Names>] , [List of String Values]]
MY CODE:
DataTable dt = null;
SqlConnection cn = null;
SqlDataReader dataReader = null;
SqlBulkCopy bulkInsert = null;
StreamReader reader = null;
string path = string.Empty;
public void test()
{
string TableIdentiferFilepath = HttpContext.Current.Server.MapPath("/testfile/TableIdentifer.txt");
Dictionary<string, string> TableIdentifer_TableName = null;
Dictionary<string, List<string>> Table_Name_ColumnName = null;
using (reader = new StreamReader(TableIdentiferFilepath))
{
TableIdentifer_TableName = new Dictionary<string, string>();
Table_Name_ColumnName = new Dictionary<string, List<string>>();
while (!reader.EndOfStream)
{
string[] curr = reader.ReadLine().Split(new string[] { ",", "\r\n" }, StringSplitOptions.None);
TableIdentifer_TableName.Add(curr[0], curr[1]);
using (cn = new SqlConnection(ConString.Connection.conn))
{
cn.Open();
if (cn.State == ConnectionState.Open)
{
string query = string.Format("select column_name from information_schema.columns where table_name = '{0}' order by ordinal_position", curr[1].ToString());
using (SqlCommand cmd = new SqlCommand(query))
{
using (SqlDataAdapter da = new SqlDataAdapter(query, cn))
{
using (dt = new DataTable())
{
da.Fill(dt);
List<string> dataColumns = dt.AsEnumerable().Select(r => r.Field<string>("column_name")).ToList();
Table_Name_ColumnName.Add(curr[1], dataColumns);
}
}
}
}
}
}
}
string path = HttpContext.Current.Server.MapPath("/TextFile/DataSample.txt");
List<KeyValuePair<string, List<string>>> KVPValues = new List<KeyValuePair<string, List<string>>>();
using (reader = new StreamReader(path))
{
while (!reader.EndOfStream)
{
string[] arr = reader.ReadLine().Split(new string[] { "|", "\r\n" }, StringSplitOptions.None);
var collValues = new List<string>();
KVPValues.Add(new KeyValuePair<string, List<string>>(arr[0], arr.Skip(1).AsEnumerable().ToList()));
foreach (var item in TableIdentifer_TableName)
{
foreach (var item2 in Table_Name_ColumnName.Where(c => c.Key == item.Value))
{
var curr_val = item2.Value;
var currKey = KVPValues.Where(p => p.Key == item.Key).ToList();
}
}
}
}
}
This is a BIG PICTURE! Hopefully now people will understand what i want to achieve.
Related
I have 2 select statements in a stored procedure and getting result set in a data reader.
My objective is to merge the data from 2 selects into a single JSON string.
I am using nested do while and at the end of the while, I am getting an error
Invalid attempt to call Read when reader is closed.
Below is the code:
try
{
con.Open();
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "SPWorkMetaData";
rd = cmd.ExecuteReader();
List<Dictionary<String, Object>> tableRow = new List<Dictionary<string, object>>();
List<Dictionary<String, Object>> tableRow1 = new List<Dictionary<string, object>>();
Dictionary<String, Object> rows;
if (rd.HasRows)
{
do
{
dt = new DataTable();
dt.Load(rd);
foreach (DataRow dr in dt.Rows)
{
rows = new Dictionary<string, object>();
int i = 1;
foreach (DataColumn col in dt.Columns)
{
rows.Add(col.ColumnName, dr[col].ToString());
i = i + 1;
}
tableRow.Add(rows);
status = "true";
}
retvalue = serializer.Serialize(tableRow).ToString();
//response = "{\"status\":\"" + status + "\",\"data\":" + retvalue + "}";
do
{
DataTable dt1 = new DataTable();
dt1.Load(rd);
foreach (DataRow dr in dt1.Rows)
{
Dictionary<String, Object> rows1 = new Dictionary<String, Object>();
int i = 1;
foreach (DataColumn col in dt1.Columns)
{
rows1.Add(col.ColumnName, dr[col].ToString());
i = i + 1;
}
tableRow1.Add(rows1);
status = "true";
}
retvalue = serializer.Serialize(tableRow).ToString() + serializer.Serialize(tableRow1).ToString();
response = "{\"status\":\"" + status + "\",\"data\":" + retvalue + "}";
}
while (rd.Read());
}
while (rd.NextResult()) ;
}
}
Does the error means to convey that SQL connection is closed, if so, then it is mentioned that I am closing connection only in the finally block.
Need to understand this, kindly provide some guidance.
Thanks.
Well, you don't note if the two tables returned are the same data structure?
and that then begs the question why a union query is not being used?
Also I don't think (doubt) you want to serialized a data row, since a data row has "extra" things like is the row dirty (been changed) and quite a few more "extra" attributes that I doubt you want to become part of the json result for the client.
So, lets pull the two tables (and ignore WHY a union query is not being used here!!!).
But, say I have two tables. tblHotels, and People - both of them have FirstName, lastname columns.
So, in theory, we need a clean data structure if we going to get a "reasonable" json string.
So, I would suggest this code:
class PersonName
{
public string FirstName = "";
public string LastName = "";
}
protected void Button1_Click(object sender, EventArgs e)
{
using (SqlConnection conn = new SqlConnection(Properties.Settings.Default.TEST4))
{
using (SqlCommand cmdSQL = new SqlCommand("Test2", conn))
{
SqlDataAdapter da = new SqlDataAdapter(cmdSQL);
conn.Open();
DataSet1 ds = new DataSet1();
da.Fill(ds);
List<PersonName> MyNames = new List<PersonName>();
// merge records form both tables (FirstName, LastName) into list
foreach (DataRow OneRow in ds.Tables[0].Rows)
{
PersonName OneName = new PersonName();
OneName.FirstName = OneRow["FirstName"].ToString();
OneName.LastName = OneRow["LastName"].ToString();
MyNames.Add(OneName);
}
// now merge 2nd table
foreach (DataRow OneRow in ds.Tables[1].Rows)
{
PersonName OneName = new PersonName();
OneName.FirstName = OneRow["FirstName"].ToString();
OneName.LastName = OneRow["LastName"].ToString();
MyNames.Add(OneName);
}
// at this point, we now have a merge of both tables
JavaScriptSerializer js = new JavaScriptSerializer();
string MyJSON = js.Serialize(MyNames);
}
}
}
my call -- I want to pass it a dictionary full of keys and values
QueryBuilder query = new QueryBuilder(new SqliteConnection(#"Data Source = D:/chinook.db"));
Dictionary<string, string> loadDictionary = new Dictionary<string, string>();
Console.WriteLine(query.Create("albums", loadDictionary)); //What do I replace loadDictionary with?
method name
public void Create(string tableName, Dictionary<string, string> columnsAndVals)
{
var commandText = $"insert into {tableName}(";
foreach (var item in columnsAndVals.Keys) // adds keys into command text
{
commandText += item + ",";
}
commandText += ") values(";
foreach (var item in columnsAndVals) // adds values into command text
{
commandText += $"#{item},";
}
var cmd = new SqliteCommand(commandText, Connection);
using (cmd)
{
foreach (var item in columnsAndVals)
{
cmd.Parameters.Add($"#{columnsAndVals.Keys}, #{columnsAndVals}");
}
cmd.ExecuteNonQuery();
}
}
I am constructing a method that will use a sql connection to insert a column and a value into a sql database.
To do this, I am using a dictionary to hold the columns and values I want to insert. (e.g. insert into (tableName)(columnsAndVals.Keys) values (columnsAndVals.Values) etc etc)
What I don't understand is how to establish a dictionary in my driver, then pass it as a parameter into my Create method. Is there a way to do this?
To create an insert query, in addition to the table name, you need to have three lists of values: column name, parameter name, and parameter value. While Dictionary<TKey, TValue allows you to pass only two lists (keys and values).
Therefore, you need to change the method signature. I would completely reject the Dictionary.
You can either pass three separate lists:
public int Create(string tableName, List<string> columnNames, List<string> parameterNames, List<object> parameterValues)
either a single list with a specially created class:
class QueryValue
{
public string ColumnName { get; set; }
public string ParameterName { get; set; }
public object ParameterValue { get; set; }
}
public int Create(string tableName, List<QueryValue> values)
{
var columnNames = string.Join(",", values.Select(v => v.ColumnName));
var parameterNames = string.Join(",", values.Select(v => v.ParameterName));
var sb = new StringBuilder()
.Append("insert into ")
.Append(tableName)
.Append('(')
.Append(columnNames)
.Append(") values(")
.Append(parameterNames)
.Append(')');
var commandText = sb.ToString();
//Console.WriteLine(commandText); // to see result
using (var cmd = new SqliteCommand(commandText, Connection))
{
foreach (var value in values)
{
cmd.Parameters.Add(value.ParameterName, value.ParameterValue);
}
return cmd.ExecuteNonQuery();
}
}
Call the method like this:
var values = new List<QueryValue>
{
new QueryValue{ ColumnName="a", ParameterName="#a", ParameterValue="aa" },
new QueryValue{ ColumnName="b", ParameterName="#b", ParameterValue="bb" },
new QueryValue{ ColumnName="c", ParameterName="#c", ParameterValue="cc" }
};
int rowsAffected = Create("albums", values);
Note that the ExecuteNonQuery method returns the number of rows affected. Therefore, our Create method returns int.
You can also use a tuple instead of a separate class.
However, I strongly oppose such methods. Their use looks rather crooked and fraught with errors.
Each table must have its own insertion method defined, as well as selections, updates, and deletions.
For example
public int InsertUser(int id, string name)
{
var commandText = "insert into User (id, name) values (#id, #name)";
using (var cmd = new SqlCommand(commandText, Connection))
{
cmd.Parameters.Add("id", id);
cmd.Parameters.Add("name", name);
return cmd.ExecuteNonQuery();
}
}
Of course, there will be many such methods, but their use is much easier and less error prone.
i want to write query which is written using linq and data dictionary i want to convert that query in ado.net form
var insData = new Dictionary<string, string>();
insData = db.Query<TableName>("select Name, Value from Table where PId={id} and type<>{(int)Enum.PElement}").ToDictionary(key => key.Name??t.TName, Val => Val.Value);
This above is the query i want to convert it into ado.net form to provide dictionary with this type
ToDictionary(key => key.Name??t.TName, Val => Val.Value);
i have written half of the code untill to fill the dataset
but further i am quite confused how to go further to convert it to dictionary with keypair value shown above.
Here is the code what i have tried
var dictonary = new SqlCommand($"select Name, Value from Table where PId={id} and type<>{(int)Enum.PElement}", con);
SqlDataAdapter sda2 = new SqlDataAdapter(dictonary);
DataSet ds2 = new DataSet();
sda2.Fill(ds2);
IEnumerable<DataRow> dataRows = ds2.Tables[0].Rows.Cast<DataRow>().ToList();
var insData = new Dictionary<string, string>();
foreach (DataRow row2 in dataRows )
{
insData.ToDictionary(key => key.Name ?? t.TName, Val => Val.Value);
}
but i am getting error like
Keypair value does not contain defination of Name and no accessible >extension method Name accepting first argument of type KeyValuepair
I want to work that query in ado.net like it is working in linq which i have shown above.
I will do something like this :
IEnumerable<DataRow> dataRows;
using (var dataSet = new DataSet())
{
using (var command = new SqlCommand($"select Name, Value from Table where PId={id} and type<>{(int) Enum.PElement}", con))
{
using (var sda2 = new SqlDataAdapter(command))
{
sda2.Fill(dataSet);
}
}
dataRows = dataSet.Tables[0].Rows.OfType<DataRow>();
}
return dataRows.ToDictionary(row => row["Name"].ToString(), row => row["Value"].ToString());
This questions had been posted several times (1,2,3,4), but I did not find one that would apply to my case.
I have a Dictionary with the structure:
public Dictionary<Int32, PhaseMag> Data;
Where PhaseMag:
public struct PhaseMag
{
public Single Magnitude;
public Single Phase;
}
Each Key value will contain two 2 values (Mag. and Phase). If you prefer to see an image.
I need to store all the content of this Dictionary in a unique Table in the DB (Microsoft SQL). Each line of the Dictionary should become one line of my DB Table. My final table will contain 3 fields, 1) Key 2) Mag and 3) Phase. For example, if I have 30 Keys, my table will contain 30 lines, one for each key.
My ideas:
Create a foreach loop based on each Key and create an insert into to the DB (one for Key->Mag and other for Key->Phase). But I don't think that this will be the best approach, especially, because my dictionary contains several thousand of lines.
So, what should be my approach to do this? I simply need to save my Dictionary into the DB, which each line of the Dic. will be one line from the DB.
Assume you're using SQL Server, here're two of many options you have:
Bulk insert, it's recommended.
public void BulkWrite(Dictionary<Int32, PhaseMag> data)
{
var dataTable = new DataTable();
dataTable.Columns.Add(new DataColumn { DataType = typeof(int), ColumnName = "Key" });
dataTable.Columns.Add(new DataColumn { DataType = typeof(Single), ColumnName = "Magnitude" });
dataTable.Columns.Add(new DataColumn { DataType = typeof(Single), ColumnName = "Phase" });
foreach (var x in data)
{
var r = dataTable.NewRow();
dataTable.Rows.Add(r);
r[0] = x.Key;
r[1] = x.Value.Magnitude;
r[2] = x.Value.Phase;
}
using (var conn = new SqlConnection("conneciton string"))
{
conn.Open();
using (var bulkCopy = new SqlBulkCopy(conn))
{
bulkCopy.BatchSize = 4000;
bulkCopy.DestinationTableName = "YorTableName";
bulkCopy.WriteToServer(dataTable);
}
}
}
Multiple inline query, executed as batch. As all your data are number, so low risk of SQL injection even using inline query.
public void InlineQueryWrite(Dictionary<Int32, PhaseMag> data)
{
using (var conn = new SqlConnection("conneciton string"))
{
conn.Open();
foreach (var bulk in data.Select((d, i) => new {d, i}).GroupBy(x => x.i % 10))
{
var sb = new StringBuilder();
foreach (var x in bulk)
{
sb.AppendFormat("Insert Into Your_Table (Key, Magnitude, Phase) Values ({0},{1},{2});", x.d.Key, x.d.Value.Magnitude, x.d.Value.Phase);
}
using (var command = conn.CreateCommand())
{
command.CommandText = sb.ToString();
command.ExecuteNonQuery();
}
}
}
}
I haven't ran/tested through the code, but they should work.
We are currently querying the Oracle DB and returning the result in the JSON format.The Query results are returning the more duplicate rows. The Controller is something like below
public HttpResponseMessage Getdetails([FromUri] string[] Column)
{
List<string> selectionStrings = new List<string>();
string connStr = ConfigurationManager.ConnectionStrings["PConnection"].ConnectionString;
using (OracleConnection dbconn = new OracleConnection(connStr))
{
DataSet userDataset = new DataSet();
var colDict = new Dictionary<string, string>()
{
{"CATEGORY", "STCD_PRIO_CATEGORY_DESCR.DESCR"},
{"SESSION_NUMBER", "STCD_PRIO_CATEGORY_DESCR.SESSION_NUM"},
{"SESSION_START_DATE","Trunc(STCD_PRIO_CATEGORY_DESCR.START_DATE)"},
{"SESSION_START_TIME","STCD_PRIO_CATEGORY_DESCR.START_DATE"}}};
foreach (string col in Column)
{
string selector = colDict[col];
selectionStrings.Add(string.Format("{0} AS {1}", selector, col));
}
string selectString = string.Join(",", selectionStrings);
var strQuery = string.Format(#"SELECT {0}
FROM
STCD_PRIO_CATEGORY_DESCR", selectString);
}}}
So if I just add Distinct in the Select statement as
var strQuery = string.Format(#"SELECT DISTINCT {0}
FROM
STCD_PRIO_CATEGORY_DESCR", selectString);
is that should be enough? Or should we be putting DISTINCT before all the Column names
Use distinct or if you are still having trouble you can move the data result into a list and use linq to select only the unique rows.
var result = myList.GroupBy(test => test.id)
.Select(grp => grp.First())
.ToList();
taken from: Select distinct using linq