Having SQL with parameter (passing through SelectCommand.Parameters.AddWithValue),
can I get easily final SQL to run it immediately? I mean, not to have unresolved parameters in the command, but rather fully resolved SQL. I can do that manually by Replace function, but I have to deal with types, etc. So just wondering if there is a way to grab the SQL statement processed by SQL engine directly (something what I can see in SQL Profiler). E.g. to have
SELECT Name FROM TABLE WHERE Id = 15
instead of
SELECT Name FROM TABLE WHERE Id = #Id
I need to log whole SQL that is runnable copy&paste&run. I can log command and parameters, but in this approach I have to manually construct the SQL statement.
DataTable dt = new DataTable();
query = "SELECT Name FROM TABLE WHERE Id = #Id";
using (SqlDataAdapter da = new SqlDataAdapter(query, String.Format(#"Data Source={0};Initial Catalog={1};Integrated Security=SSPI", relServer, relDatabase)))
{
if (QueryParams != null && QueryParams.Count > 0)
{
foreach (KeyValuePair<string, object> entry in QueryParams)
{
da.SelectCommand.Parameters.AddWithValue(entry.Key, entry.Value ?? DBNull.Value);
}
}
da.Fill(dt);
}
Related
In my C# project, I am executing a query against a SQL 2014 database to retrieve Employee data, including a PeriodEnd which is stored in the database as a Decimal (i.e., Nov 17, 2016 is stored as 20161117). The database I am querying is not our product so I cannot change the type to a DateTime field.
Here is the SQL script being executed:
SELECT DISTINCT
e.EMPLOYEE as EmpNo,
ch.PEREND As PeriodEnd,
ch.PRPOSTSTAT
FROM UPEMPL e
INNER JOIN UPCHKH ch
ON e.EMPLOYEE = ch.EMPLOYEE
WHERE
ch.PEREND = #PERIODEND
Here is the SqlDataAdapter call:
ExecuteSqlCommandScript(String sqlScript, List<SqlParams> sqlParams)
{
. . . (setup SqlConnection info)
using (SqlConnection _conn = new SqlConnection(connectionString))
{
using (SqlCommand _cmd = new SqlCommand())
{
_cmd.CommandText = sqlScript;
_cmd.Connection = _conn;
_cmd.Connection.Open();
// add SqlParameters to SQL command
if (sqlParams != null)
{
_cmd.Parameters.AddRange(sqlParams.ToArray());
}
using (SqlDataAdapter _sqlDataAdapter = new SqlDataAdapter(_cmd))
{
try
{
// Save Table info to Results object
DataSet _dataSet = new DataSet();
_sqlDataAdapter.Fill(_dataSet);
SqlResult _result = new SqlResult();
_result.DataSet = _dataSet;
_result.TableCount = _dataSet.Tables.Count;
this.Results.Add(_result);
}
}
}
}
}
Using SQL Server Profiler I can see the query passed to SQL is:
exec sp_executesql N'
SELECT DISTINCT
e.EMPLOYEE as EmpNo,
ch.PEREND As PeriodEnd,
ch.PRPOSTSTAT
FROM UPEMPL e
INNER JOIN UPCHKH ch
ON e.EMPLOYEE = ch.EMPLOYEE
WHERE
ch.PEREND = #PERIODEND
',N'#PERIODEND nvarchar(8)',#PERIODEND=N'20161101'
If I run this directly in SQL these are the reults:
However, the results of the DataTable created by the _sqlDataAdapter is:
Is there a way to force SqlDataAdapter to use the data type as it is in the results? Or is it possible that SQL is indeed returning a DateTime object instead of a Decimal (the PeriodEnd column is defined in SQL as decimal(9,0))? If so is there a reason for this and/or a way to prevent it?
Try and add the column type explicitly when you are creating the dataset , by adding datacolumn of type decimal to the datatable.
My guess is that since you are not explicitly specifying the column type, .net is deriving it from the value.
Thank you all for your help in this. It was the suggestion to look at the _cmd object that lead me to the solution. There was a piece of code that was supposed to change the database to use the database ABC. This piece of code was not working properly and so the database connection stayed on database XYZ. What made this so difficult to resolve is that database XYZ has views defined that return data from databse ABC, but with the PeriodEnd column as a DateTime and not a decimal. I did not know about these views and when I saw data that matched what I expected, I didn't even think to look and make sure the database was correctly assigned.
I would like to insert all the id's in a sql table. The following way works but this take very long. What is the best or better way to do this to increase the speed.
using (SqlConnection connection = new SqlConnection(ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString))
{
string query = "";
foreach (var id in ids) // count = 60000
{
{
query += "INSERT INTO [table] (id) VALUES (" + id + ");";
}
}
SqlCommand command = new SqlCommand(query, connection);
connection.Open();
using (SqlDataReader reader = command.ExecuteReader())
{
reader.Close();
}
connection.Close();
}
You can use the SqlBulkCopy to insert large amounts of data - something like this:
// define a DataTable with the columns of your target table
DataTable tblToInsert = new DataTable();
tblToInsert.Columns.Add(new DataColumn("SomeValue", typeof (int)));
// insert your data into that DataTable
for (int index = 0; index < 60000; index++)
{
DataRow row = tblToInsert.NewRow();
row["SomeValue"] = index;
tblToInsert.Rows.Add(row);
}
// set up your SQL connection
using (SqlConnection connection = new SqlConnection(ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString))
{
// define your SqlBulkCopy
SqlBulkCopy bulkCopy = new SqlBulkCopy(connection);
// give it the name of the destination table WHICH MUST EXIST!
bulkCopy.DestinationTableName = "BulkTestTable";
// measure time needed
Stopwatch sw = new Stopwatch();
sw.Start();
// open connection, bulk insert, close connection
connection.Open();
bulkCopy.WriteToServer(tblToInsert);
connection.Close();
// stop time measurement
sw.Stop();
long milliseconds = sw.ElapsedMilliseconds;
}
On my system (PC, 32GB RAM, SQL Server 2014) I get those 60'000 rows inserted in 135 - 185 milliseconds.
Consider Table-Valued Parameters. They are an easy way to send a batch of data into a stored procedure that will then handle them on the SQL side, and they aren't restricted in most of the other approaches you will see are (insert limits, etc).
In the database create a custom Type that has the schema of your table.
CREATE TYPE dbo.TableType AS TABLE
( ID int )
Create a DataTable that matches your table schema (including column name and order).
DataTable newTableRecords = new DataTable();
// Insert your records, etc.
Create a stored procedure that receives a table parameter, and inserts the records from that parameter into your real table.
CREATE PROCEDURE usp_InsertTableRecords
(#tvpNewTableRecords dbo.TableType READONLY)
AS
BEGIN
INSERT INTO dbo.Table(ID)
SELECT tvp.ID FROM #tvpNewTableRecords AS tvp;
END
Call the procedure from your application code, passing in your data table as a parameter.
using (connection)
{
// Configure the SqlCommand and SqlParameter.
SqlCommand insertCommand = new SqlCommand(
"usp_InsertTableRecords", connection);
insertCommand.CommandType = CommandType.StoredProcedure;
SqlParameter tvpParam = insertCommand.Parameters.AddWithValue(
"#tvpNewTableRecords", newTableRecords);
tvpParam.SqlDbType = SqlDbType.Structured;
// Execute the command.
insertCommand.ExecuteNonQuery();
}
I've had really great performance at very large volumes with this approach, and it is nice because it allows everything to be set-based without any arbitrary insert limits like the INSERT INTO (Table) VALUES (1),(2),(3)... approach.
I have a requirement where I need to read queries from Access DB in c# and check if the access db query has any keyword like "KEY" if it has keywords I need to enclose that in square brackets"[]".just like how it is done in SQL.
Could someone suggest me how to do that?
You can retrieve the query text like this:
string connString = #"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\...\myDB.mdb";
using (var conn = new OleDbConnection(connString )) {
conn.Open();
string[] restrictions = new string[] { null, null, "myQuery" };
DataTable schema = conn.GetSchema("Views", restrictions);
if (schema.Rows.Count > 0) {
DataRow row = schema.Rows[0];
string queryText = (string)row["VIEW_DEFINITION"];
Console.WriteLine(queryText);
}
}
If you drop the restrictions argument with the query name, conn.GetSchema("Views") returns one row for each query. If you query conn.GetSchema("Procedures") other types of queries like insert, update and DDL statements that are not considered as queries are returned in row["PROCEDURE_DEFINITION"].
View (query) names are returned in row["TABLE_NAME"] and procedure names in row["PROCEDURE_NAME"].
And you can update the query like this:
using (var conn = new OleDbConnection(connString)) {
conn.Open();
var cmd = new OleDbCommand("DROP PROCEDURE myQuery", conn);
cmd.ExecuteNonQuery();
cmd = new OleDbCommand("CREATE PROCEDURE myQuery AS SELECT * FROM myTable", conn);
cmd.ExecuteNonQuery();
}
Strangely enough the OleDb CREATE DDL (Data Definition Language) designates the queries as 'procedures' but the schema table returns a 'VIEW_DEFINITION' and the query name is returned in the column 'TABLE_NAME'. SELECT queries must be retrieved as "Views", other types of queries as "Procedures"; however, both types are created as PROCEDUREs.
While I was testing the answer that #Olivier Jacot-Descombes provided, I was not able to retreive all the queries text representation. Therefore I applied some other method where you open the existing Ms Access database instance and read the queries that are stored in it.
Here is the class I used:
public class MsAccess
{
private Microsoft.Office.Interop.Access._Application _oAccess;
public MsAccess(string path)
{
_oAccess = (Microsoft.Office.Interop.Access._Application)System.Runtime.InteropServices.Marshal.BindToMoniker(path);
}
public string ReturnSqlQueryText(string queryName)
{
string queryDef = null;
var qdefs = _oAccess.CurrentDb().QueryDefs;
foreach (QueryDef qdef in qdefs)
{
if(qdef.Name.Equals(queryName))
queryDef = qdef.SQL;
}
return queryDef;
}
}
Using this code might require you adding using Microsoft.Office.Interop.Access.Dao and Microsoft.Office.Interop.Access both (15.0.0.0) where you can find them under Extension on the reference menu
I'm working with 2 SQL 2008 Servers on different machines. The server names are source.ex.com, and destination.ex.com.
destination.ex.com is linked to source.ex.com and the appropriate permissions are in place for source.ex.com to write to a database called bacon-wrench on destination.ex.com
I've logged into source.ex.com via SMS and tested this query (successfully):
INSERT INTO [destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]
(PunchID, BaconID) VALUES (4,6);
In a C# .NET 4.0 WebPage I connect to source.ex.com and perform a similar query (successfully):
using(SqlConnection c = new SqlConnection(ConfigurationManager.ConnectionStrings["SOURCE"].ConnectionString))
{
c.Open();
String sql = #"
INSERT INTO [destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]
(PunchID, BaconID) VALUES (34,56);";
using(SqlCommand cmd = new SqlCommand(sql, c))
{
cmd.ExecuteNonQuery();
}
}
For small sets of insert statements (say 20 or less) doing something like this performs fine:
using(SqlConnection c = new SqlConnection(ConfigurationManager.ConnectionStrings["SOURCE"].ConnectionString))
{
c.Open();
String sql = #"
INSERT INTO [destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]
(PunchID, BaconID) VALUES (34,56);
INSERT INTO [destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]
(PunchID, BaconID) VALUES (22,11);
INSERT INTO [destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]
(PunchID, BaconID) VALUES (33,55);
INSERT INTO [destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]
(PunchID, BaconID) VALUES (1,2);";
using(SqlCommand cmd = new SqlCommand(sql, c))
{
cmd.ExecuteNonQuery();
}
}
I'm trying to do something like this with around 20000 records. The above method takes 11 minutes to complete -- which I assume is the server sreaming at me to make it some kind of bulk operation. From other StackOverflow threads the SqlBulkCopy class was recommended and it takes as a parameter DataTable, perfect!
So I build a DataTable and attempt to write it to the server (fail):
DataTable dt = new DataTable();
dt.Columns.Add("PunchID", typeof(int));
dt.Columns.Add("BaconID", typeof(int));
for(int i = 0; i < 20000; i++)
{
//I realize this would make 20000 duplicate
//rows but its not important
dt.Rows.Add(new object[] {
11, 33
});
}
using(SqlConnection c = new SqlConnection(ConfigurationManager.ConnectionStrings["SOURCE"].ConnectionString))
{
c.Open();
using(SqlBulkCopy bulk = new SqlBulkCopy(c))
{
bulk.DestinationTableName = "[destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]";
bulk.ColumnMappings.Add("PunchID", "PunchID");
bulk.ColumnMappings.Add("BaconID", "BaconID");
bulk.WriteToServer(dt);
}
}
EDIT2: The below message is what I'm attempting to fix:
The web page crashes at bulk.WriteToServer(dt); with an error message Database bacon-wrench does not exist please ensure it is typed correctly. What am I doing wrong? How do I change this to get it to work?
EDIT1:
I was able to speed up the query significantly using the below syntax. But it is still very slow for such a small record set.
using(SqlConnection c = new SqlConnection(ConfigurationManager.ConnectionStrings["SOURCE"].ConnectionString))
{
c.Open();
String sql = #"
INSERT INTO [destination.ex.com].[bacon-wrench].[dbo].[tblFruitPunch]
(PunchID, BaconID) VALUES
(34,56),
(22,11),
(33,55),
(1,2);";
using(SqlCommand cmd = new SqlCommand(sql, c))
{
cmd.ExecuteNonQuery();
}
}
If you are using SQL Server 2008+, you can introduce a Table user datatype. Prepare the type, receiving table and stored procedure something like below. Data type and stored procedure is on the local system. I generally have an if statement in the code detecting whether the table is remote or local, remote I do this, local I use SqlBulkCopy.
if(TYPE_ID(N'[Owner].[TempTableType]') is null)
begin
CREATE TYPE [Owner].[TempTableType] AS TABLE ( [PendingID] uniqueidentifier, [Reject] bit)
end
IF NOT EXISTS (SELECT * FROM [LinkedServer].[DatabaseOnLS].sys.tables where name = 'TableToReceive')
EXEC('
CREATE TABLE [DatabaseOnLS].[Owner].[TableToReceive] ( [PendingID] uniqueidentifier, [Reject] bit)
') AT [LinkedServer]
else
EXEC('
TRUNCATE TABLE [DatabaseOnLS].[Owner].[TableToReceive]
') AT [LinkedServer]
CREATE PROCEDURE [Owner].[TempInsertTable]
#newTableType TempTableType readonly
AS
BEGIN
insert into [LinkedServer].[DatabaseOnLS].[Owner].[TableToReceive] select * from #newTableType
END
In the C# code you can then do something like this to insert the DataTable into the table on the linked server (I'm using an existing UnitOfWork, which already have a connection and transaction):
using (var command = new SqlCommand("TempInsertTable",
oUoW.Database.Connection as SqlConnection) { CommandType = CommandType.StoredProcedure }
)
{
command.Transaction = oUoW.Database.CurrentTransaction as SqlTransaction;
command.Parameters.Add(new SqlParameter("#newTableType", oTempTable));
drResults = command.ExecuteReader();
drResults.Close();
}
After trying a number of things including linked server settings, collations, synonyms, etc., I eventually got to this error message:
Inserting into remote tables or views is not allowed by using the BCP utility or by using BULK INSERT.
Perhaps you can bulk insert to a staging table on your local server (your code works fine for this) and then insert from that staging table to your linked server from there, followed by a local delete of the staging table. You'll have to test for performance.
I am working on sql server monitoring product and i have database query that will fetch data regarding All Table details of all the Databases in SQL server.
For this i have two options.
Fire query on data base from code as select name from [master].sys.sysdatabases
Get the DB name of all the data base first then i will fire my main query on each DB
using "USE <fetched DB name>;"+"mainQuery";
Please check followin code for the same.
public DataTable GetResultsOfAllDB(string query)
{
SqlConnection con = new SqlConnection(_ConnectionString);
string locleQuery = "select name from [master].sys.sysdatabases";
DataTable dtResult = new DataTable("Result");
SqlCommand cmdData = new SqlCommand(locleQuery, con);
cmdData.CommandTimeout = 0;
SqlDataAdapter adapter = new SqlDataAdapter(cmdData);
DataTable dtDataBases = new DataTable("DataBase");
adapter.Fill(dtDataBases);
foreach (DataRow drDB in dtDataBases.Rows)
{
if (dtResult.Rows.Count >= 15000)
break;
locleQuery = " Use [" + Convert.ToString(drDB[0]) + "]; " + query;
cmdData = new SqlCommand(locleQuery, con);
adapter = new SqlDataAdapter(cmdData);
DataTable dtTemp = new DataTable();
adapter.Fill(dtTemp);
dtResult.Merge(dtTemp);
}
return dtResult;
}
I will use sys store procedure i.e.EXEC sp_MSforeachdb and fetched data will be stored store data in table datatype select from temptable; Drop Table temptable.
Check following query for the same
Declare #TableDetail table
(
field1 varchar(500),
field2 int,
field3 varchar(500),
field4 varchar(500),
field5 decimal(18,2),
field6 decimal(18,2)
)
INSERT #TableDetail EXEC sp_MSforeachdb 'USE [?]; QYERY/COMMAND FOR ALL DATABASE'
Select
field1,field2 ,field3 ,field4 ,field5,field6 FROM #TableDetail
Note : In second option query takes time because if number of database and number of table are huge then this will wait until all database get finish.
Now my question is which is the good option from above two options and why? or any other solution for the same.
Thanks in advance.
One key difference is the second option blocks until everything is done. All of the work is done sql server side. That has the issue of not being able to apply feedback to the user as it runs and it can potentially time out and not be resiliant to network blips. This option can be used as a pure sql script (some sql admins like that) where the first needs a program.
In the first example, the client is doing iterative more granular tasks where you can supply feedback to the user. You can also retry in the face of network blips without redoing all of the work. In the first example, you can also use SqlConnectionBuild instead of USE concatentation.
If performance is a concern, you could also potentially parallelize the first one with some locking around adapter.Fill
Both suck - they are both serial.
Use the first, get rid of the ridiculous objects (DataSet) and use TASKS to parallelize X databases at the same time. X determined by trying ut how much load the server can handle.
Finished.
If your queries are simple enough you can try to generate single script instead of execute queries in each DB one by one:
select 'DB1' as DB, Field1, Field2, ...
from [DB1]..[TableOrViewName]
union all
select 'DB2' as DB, Field1, Field2, ...
from [DB2]..[TableOrViewName]
union all
...
Everything is looking fine. I just want to add Using statements for IDisposable objects
public DataTable GetResultsOfAllDB(string query)
{
using (SqlConnection con = new SqlConnection(_ConnectionString))
{
string locleQuery = "select name from [master].sys.sysdatabases";
DataTable dtResult = new DataTable("Result");
using (SqlCommand cmdData = new SqlCommand(locleQuery, con))
{
cmdData.CommandTimeout = 0;
using (SqlDataAdapter adapter = new SqlDataAdapter(cmdData))
{
using (DataTable dtDataBases = new DataTable("DataBase"))
{
adapter.Fill(dtDataBases);
foreach (DataRow drDB in dtDataBases.Rows)
{
if (dtResult.Rows.Count >= 15000)
break;
locleQuery = " Use [" + Convert.ToString(drDB[0]) + "]; " + query;
cmdData = new SqlCommand(locleQuery, con);
adapter = new SqlDataAdapter(cmdData);
using (DataTable dtTemp = new DataTable())
{
adapter.Fill(dtTemp);
dtResult.Merge(dtTemp);
}
}
return dtResult;
}
}
}
}
}