I need to do a full outer join on 2 datatables dinamically, I don't know what columns are in the datatables but they will be the same in each table, I also only have the name of the column I need to do the join on in a variable. Is there a way of doing this?
What I need to do is join 2 datatables in a C# script. I'm using a Dataflow in an SSIS to get data from a couple of files, and at the end I need to compare the 2 final sets of data. I need to to this on whatever 2 datatables as long as they have the same columns, so I can't finish the process in an SSIS as I need to specify the columns.
The GetData() I just use it in case I need to compare 2 tables but donnesPROD and donnesDEV are filled from object variables in the SSIS.
Here's my code so far :
DataTable donnesPROD = GetData(connectionPROD, sql_request);
DataTable donnesDEV = GetData(connectionDEV, sql_request);
Here's the code for GetData :
DataTable GetData(string cs, string query)
{
OleDbConnection conn = new OleDbConnection(cs);
conn.Open();
OleDbCommand cmd = new OleDbCommand(query, conn);
DataTable dt = new DataTable();
dt.Load(cmd.ExecuteReader());
conn.Close();
return dt;
}
I have the list of the columns in another datatable, and I have the name of the primary key in a string variable key. From here I need to be able to do a fullouterjoin of donnesPROD and donnesDEV on key. Can this be done this way? Or is there a way of generating the script code it self dynamically and then execute it?
You have two options.
Conditional joins
If you don't know the specific column name, but you do have some idea what the column name might be, you could do a conditional join like this:
CREATE PROCEDURE ExampleDynamicJoin(#JoinColumn AS VarChar(40))
AS
BEGIN
SELECT *
FROM TableA
JOIN TableB ON (#JoinColumn = 'ColumnA' AND TableA.ColumnA = TableB.ColumnA)
OR (#JoinColumn = 'ColumnB' AND TableA.ColumnB = TableB.ColumnB)
OR (#JoinColumn = 'ColumnC' AND TableA.ColumnC = TableB.ColumnC)
END
You may not get the best performance out of this (the conditional joins will confuse the query engine and it may not pick the best index, if it picks one at all). If the table is very large you could also do something like this. It is a bit painful-looking but will get better performance:
CREATE PROCEDURE ExampleDynamicJoin(#JoinColumn AS VarChar(40))
AS
BEGIN
IF (#JoinColumn = 'ColumnA') BEGIN
SELECT *
FROM TableA
JOIN TableB ON TableA.ColumnA = TableB.ColumnA
END
IF (#JoinColumn = 'ColumnB') BEGIN
SELECT *
FROM TableA
JOIN TableB ON TableA.ColumnB = TableB.ColumnB
END
IF (#JoinColumn = 'ColumnC') BEGIN
SELECT *
FROM TableA
JOIN TableB ON TableA.ColumnC = TableB.ColumnC
END
END
If TableA or TableA are part of a larger query, and you'd end up duplicating tons of SQL, you could always extract the resultset for just TableA and TableB into a temporary table, then use the temporary table in the larger query.
Dynamic SQL
If you don't have the foggiest about the column name and there are tons of possibilities, you could construct the SQL as a string and join that way. You should validate the column name that is passed in; not only will that make sure the column actually exists, but it will prevent the dynamic SQL from being constructed when #JoinColumn contains an injection attack, since legal column names do not contain SQL statements. Example:
CREATE PROCEDURE ExampleDynamicJoin(#JoinColumn AS VarChar(40))
AS
BEGIN
DECLARE #Sql AS VarChar(MAX)
IF NOT EXISTS
(
SELECT 0
FROM syscolumns c
JOIN sysobjects o ON o.id = c.id
WHERE o.Name = 'TableA'
AND c.Name = #JoinColumn
)
RAISERROR (15600,-1,-1, 'ExampleDynamicJoin'); //Throw error if column doesn't exist
SET #Sql =
'SELECT *
FROM TableA
JOIN TableB ON TableA.' + #JoinColumn + ' = TableB.' + #JoinColumn
sp_ExecuteSql #Sql
END
Or, if you don't use stored procedures,
DataTable ExampleDynamicJoin(string joinColumn)
{
if (!ValidateColumn(joinColumn)) throw new ArgumentException();
var sql = String.Format(
#"SELECT *
FROM TableA
JOIN TableB ON TableA.{0} = TableB.{0}",
joinColumn
);
using (var connection = GetConnectionFromSomewhere())
{
using (var cmd = new SqlCommand
{
CommandText = sql,
CommandType = CommandType.Text,
Connection = connection
})
{
var reader = cmd.ExecuteReader();
var table = new DataTable();
table.Load(reader);
return table;
}
}
}
When using dynamic SQL you should always use parameters if possible. But you can't use parameters as a column name, so in this case you have to concatenate. When concatenating, ALWAYS white list the inputs. That is why I included a function named ValidateColumn which could look like this:
bool ValidateColumn(string columnName)
{
switch columnName.ToUpper()
{
case "COLUMNA":
case "COLUMNB":
case "COLUMNC":
return true;
default:
return false;
}
}
Related
I am trying to use a two line query to a mysql database like this with C#:
set #var = 1; select id from table where id = #var`;
Executing this in C# does not give any error, the reader has columns, but don't have rows in it, so, no data is retrieved.
Running the same queries in the workbench retrieve the expected data.
Running one query to calculate the variable and other to use it replaced works.
I want to use a variable to store references and use it to filter queries that provide an union.
SET #my_value = (select min(id) from (select id from datatable limit 2000) as a);
and a complex select query, that i can resume in:
select * from
datatable as a
left join (
(select databalt2 where id > #my_value) as ba union
(select datatable3 where id > #my_value) as bb) as b
on a.id = b.id
where a.id > #my_value;
The datareader should obtain the results, but instead reports that it has no rows, but has the correct amount of columns.
And, obviously, shows no results, where it should.
Edit 1: C# code
string query = #"
SET #`my_value` = (select min(id) from (select id from datatable limit 2000) as a);
select * from
datatable as a
left join (
(select databalt2 where id > #`my_value`) as ba union
(select datatable3 where id > #`my_value`) as bb) as b
on a.id = b.id
where a.id > #`my_value`;
";
try
{
MySqlCommand command = Connection.CreateCommand();
MySqlDataReader reader;
command.CommandTimeout = commandTimeout;
command.CommandText = query;
reader = command.ExecuteReader();
}
catch (Exception ex)
{
}
These queries using local variables are possible, the local variable has to be inside like #`var`, because it can be identifyied as a local parameter.
I was missing a in a local variable call. So ... a problem between the screen and the chair!
I have an table name dbo.EmpInfo having 4 columns
1-UserId 2-SubUserId 3-Year 4-Status
I have an another table (in other database) name dbo.EmpInfo1 having 4 columns
1-UserId 2-SubUserId 3-Year 4-Status
UserId may be repeating in both tables..
Now i have to find those UserId from Both tables whose Status="Success" and this status count is < 10 and bind these values in Gridview..
for ex-I have an UserId say mayank#gmail.com and in dbo.EmpInfo it has status count=5(Status="Success") and in dbo.EmpInfo1 it has status count=7 so from both tables the total count for mayank#gmail.com is 12 so we have to bind this userId in Gridview. and Gridview having all the above columns..
i have a procedure -
ALTER proc [dbo].[sp_countUserDetails]
as
begin try
begin transaction
Select distinct(UserId) from EmpInfo where Status='Success'
union all
Select distinct(UserId) from MyDB.dbo.EmpInfo1 where Status='Sucess'
commit transaction
end try
in my .cs file i used
SqlDataReader dr = ms.sp_SelectExecuter("sp_countUserDetails");
DataTable dt = new DataTable();
dt.Load(dr);
foreach (DataRow DR in dt.Rows)
{
ms = new MethodStore();
ms.sp_SelectExecuter("sp_usercount", "#userid", (DR["UserId"]).ToString());
}
and the Procedure is-
ALTER Procedure [dbo].[sp_usercount]
#userid varchar(50)
as
declare #count1213 dec =0, #count1314 dec =0;
begin try
begin transaction
select #count1314= count(UserId) from EmpInfo where Status='Status' and UserID=#userid;
select #count1213= count(UserId) from MyDB.dbo.EmpInfo1 where Status='Success' and UserID=#userid;
select #count1213+#count1314 as 'Count'
if((#count1213+#count1314)>=10)
insert into MyTaxCafe.dbo.demo values (#userid);
commit transaction
end try
bt the table dbo.demo doesn't contain distinct UserId..because our Procedure
[dbo].[sp_countUserDetails]
give Distinct values from both table but at due to Union there is an redundancy can we control it because Same UserId may be Exist in both tables
First you need a SQL to query two database , you may try it like this
SELECT Status FROM [database1].[dbo].[TableName] AS t1 INNER JOIN [database2].[dbo].[TableName] AS t2
ON (t1.UserId = t2.UserId)
WHERE Status='Success'
GROUP BY Status
Having COUNT(Status) < 10
in your C# code , use SqlDataAdapter to fill DataTable,
then set DataGridView's field DataSource = DataTable
You must use union from your query to get expected result.
Your columns are same in both tables so u can use union without any changes
for example
select * (select * from TableName1
where Status = 'Success'
union
select * from TableName2
where Status = 'Success'
) A
where count(Status)<10
group by SubUserId, Year, Status
try this.
I'm using Entity Framework to get the total row count for a table. I simply want the row count, no where clause or anything like that. The following query works, but is slow. It took about 7 seconds to return the count of 4475.
My guess here is that it's iterating through the entire table, just like how IEnumerable.Count() extension method works.
Is there a way I can get the total row count "quickly"? is there a better way?
public int GetLogCount()
{
using (var context = new my_db_entities(connection_string))
{
return context.Logs.Count();
}
}
You can even fire Raw SQL query using entity framework as below:
var sql = "SELECT COUNT(*) FROM dbo.Logs";
var total = context.Database.SqlQuery<int>(sql).Single();
That is the way to get your row count using Entity Framework. You will probably see faster performance on the second+ queries as there is an initialization cost the first time that you run it. (And it should be generating a Select Count() query here, not iterating through each row).
If you are interested in a faster way to get the raw row count in a table, then you might want to try using a mini ORM like Dapper or OrmLite.
You should also make sure that your table is properly defined (at the very least, that it has a Primary Key), as failure to do this can also affect the time to count rows in the table.
If you have access to do so, it would be much quicker to query the sys tables to pull this information.
E.g.
public Int64 GetLogCount()
{
var tableNameParam = new SqlParameter("TableName", "Logs");
var schemaNameParam = new SqlParameter("SchemaName", "dbo");
using (var context = new my_db_entities(connection_string))
{
var query = #"
SELECT ISNULL([RowCount],0)
FROM (
SELECT SchemaName,
TableName,
Sum(I.rowcnt) [RowCount]
FROM sysindexes I
JOIN sysobjects O (nolock) ON I.id = o.id AND o.type = 'U'
JOIN (
SELECT so.object_id,
ss.name as SchemaName,
so.name as TableName
FROM sys.objects SO (nolock)
JOIN sys.schemas SS (nolock) ON ss.schema_id = so.schema_id
) SN
ON SN.object_id = o.id
WHERE I.indid IN ( 0, 1 )
AND TableName = #TableName AND SchemaName = #SchemaName
GROUP BY
SchemaName, TableName
) A
";
return context.ExecuteStoreQuery<Int64>(query, tableNameParam, schemaNameParam).First();
}
}
I have a Gridview in front end where Grid have two columns : ID and Order like this:
ID Order
1 1
2 2
3 3
4 4
Now user can update the order like in front end Gridview:
ID Order
1 2
2 4
3 1
4 3
Now if the user click the save button the ID and order data is being sent to Stored Procedure as #sID = (1,2,3,4) and #sOrder = (2,4,1,3)
Now if I want to update the order and make save I want to store it into database. Through Stored procedure how can update into the table so that the table is updated and while select it gives me the results like:
ID Order
1 2
2 4
3 1
4 3
There is no built in function to parse these comma separated string. However, yo can use the XML function in SQL Server to do this. Something like:
DECLARE #sID VARCHAR(100) = '1,2,3,4';
DECLARE #sOrder VARCHAR(10) = '2,4,1,3';
DECLARE #sIDASXml xml = CONVERT(xml,
'<root><s>' +
REPLACE(#sID, ',', '</s><s>') +
'</s></root>');
DECLARE #sOrderASXml xml = CONVERT(xml,
'<root><s>' +
REPLACE(#sOrder, ',', '</s><s>') +
'</s></root>');
;WITH ParsedIDs
AS
(
SELECT ID = T.c.value('.','varchar(20)'),
ROW_NUMBER() OVER(ORDER BY (SELECT 1)) AS RowNumber
FROM #sIDASXml.nodes('/root/s') T(c)
), ParsedOrders
AS
(
SELECT "Order" = T.c.value('.','varchar(20)'),
ROW_NUMBER() OVER(ORDER BY (SELECT 1)) AS RowNumber
FROM #sOrderASXml.nodes('/root/s') T(c)
)
UPDATE t
SET t."Order" = p."Order"
FROM #tableName AS t
INNER JOIN
(
SELECT i.ID, p."Order"
FROM ParsedOrders p
INNER JOIN ParsedIDs i ON p.RowNumber = i.RowNumber
) AS p ON t.ID = p.ID;
Live Demo
Then you can put this inside a stored procedure or whatever.
Note that: You didn't need to do all of this manually, it should be some way to make this gridview update the underlying data table automatically through data binding. You should search for something like this instead of all this pain.
You could use a table valued parameter to avoid sending delimiter-separated values or even XML to the database. To do this you need to:
Declare a parameter type in the database, like this:
CREATE TYPE UpdateOrderType TABLE (ID int, Order int)
After that you can define the procedure to use the parameter as
CREATE PROCEDURE UpdateOrder (#UpdateOrderValues UpdateOrderType readonly)
AS
BEGIN
UPDATE t
SET OrderID = tvp.Order
FROM <YourTable> t
INNER JOIN #UpdateOrderValues tvp ON t.ID=tvp.ID
END
As you can see, the SQL is trivial compared to parsing XML or delimited strings.
Use the parameter from C#:
using (SqlCommand command = connection.CreateCommand()) {
command.CommandText = "dbo.UpdateOrder";
command.CommandType = CommandType.StoredProcedure;
//create a table from your gridview data
DataTable paramValue = CreateDataTable(orderedData)
SqlParameter parameter = command.Parameters
.AddWithValue("#UpdateOrderValues", paramValue );
parameter.SqlDbType = SqlDbType.Structured;
parameter.TypeName = "dbo.UpdateOrderType";
command.ExecuteNonQuery();
}
where CreateDataTable is something like:
//assuming the source data has ID and Order properties
private static DataTable CreateDataTable(IEnumerable<OrderData> source) {
DataTable table = new DataTable();
table.Columns.Add("ID", typeof(int));
table.Columns.Add("Order", typeof(int));
foreach (OrderData data in source) {
table.Rows.Add(data.ID, data.Order);
}
return table;
}
(code lifted from this question)
As you can see this approach (specific to SQL-Server 2008 and up) makes it easier and more formal to pass in structured data as a parameter to a procedure. What's more, you're working with type safety all the way, so much of the parsing errors that tend to crop up in string/xml manipulation are not an issue.
You can use charindex like
DECLARE #id VARCHAR(MAX)
DECLARE #order VARCHAR(MAX)
SET #id='1,2,3,4,'
SET #order='2,4,1,3,'
WHILE CHARINDEX(',',#id) > 0
BEGIN
DECLARE #tmpid VARCHAR(50)
SET #tmpid=SUBSTRING(#id,1,(charindex(',',#id)-1))
DECLARE #tmporder VARCHAR(50)
SET #tmporder=SUBSTRING(#order,1,(charindex(',',#order)-1))
UPDATE dbo.Test SET
[Order]=#tmporder
WHERE ID=convert(int,#tmpid)
SET #id = SUBSTRING(#id,charindex(',',#id)+1,len(#id))
SET #order=SUBSTRING(#order,charindex(',',#order)+1,len(#order))
END
I need to execute a query like as given below which contain a dataset in it.
UPDATE <OrderDataset>
SET FKProduct = P.PKProduct
FROM <OrderDataset> DS
INNER JOIN tblCustomer C ON DS.FKCustomer = C.PKCustomer
INNER JOIN tblProduct P ON C.PKCustomer = P.FKCustomer
INNER JOIN tblStock S ON S.FKProduct = P.PKProduct
AND DS.RotationNumber = S.RotationNumber
AND ISNULL(DS.RotationLineNo,'NULL') = ISNULL(S.RotationLineNo,'NULL')
WHERE DS.FKProduct IS NULL
Is there any way to implemnt this.
There is no direct way to do this.
You can pass the table inside the dataset as a parameter to sql server stored procedure ans then do the jon in sp. .
how to pass data table as parameter to sp