I having XMl file I am Reading All the Xml using this code . But mean while I want to insert XML records to sql table,Whatever I have Read. I did not Find any Solution for that .XMl consists No of rows and how can i insert in Proper way .
if (File.Exists(xmlpath))
{
try
{
XDocument xmlDoc = XDocument.Load(xmlpath);
var vrresult = from a in xmlDoc.XPathSelectElements("/Parts/Part")
select new
{
S = a.Element("Section").Value,
M = a.Element("Mark").Value,
Q = a.Element("Qty").Value,
W = a.Element("Weight").Value,
RD = a.Element("databaseUpdateMark").Value
};
GridView1.DataSource = vrresult;
GridView1.DataBind();
}
catch (Exception ex)
{
Lbmsg.Text = ex.Message;
}
finally
{
}
}
here is one example how to use OPENXML function
Bulk Insertion of Data Using C# DataTable and SQL server OpenXML function
Make use of function : OPENXML (Transact-SQL) which is avialble in sql server allows you to insert data in server....
also read : Use OPENXML to insert values from a string of XML
Example :
--Use OPENXML to pull records from an XML string and insert them into a database
DECLARE #idoc int
DECLARE #doc varchar (1000)
--XML document
SET #doc ='
<ROOT>
<Employee>
<FirstName>Tom</FirstName>
<LastName>Jones</LastName>
</Employee>
<Employee>
<FirstName>Gus</FirstName>
<LastName>Johnson</LastName>
</Employee>
</ROOT>'
--Create an internal representation of the XML document
--the idoc variable is set as the document handler, so we can refer to it later
EXEC sp_xml_preparedocument #idoc OUTPUT, #doc
-- Use the OPENXML rowset provider with an Insert
-- #idoc lets us know which internal representation of an xml doc to use
-- '/ROOT/Employee' shows us which node in the xml tree to work on
-- the 2 denotes we want to use elemenet centric mapping for the values, as oppsed to attributes or a combination of both
-- in the 'With' we specify how the output rowset will look
INSERT INTO Employees (FirstName, LastName)
SELECT *
FROM OPENXML (#idoc, '/ROOT/Employee',2)
WITH (FirstName varchar(10),
LastName varchar(20)
)
-- Clear the XML document from memory
EXEC sp_xml_removedocument #idoc
Related
I have a table in SQL Server. One of the columns in this table holds XML data (ntext) like this:
<data>
<id>123</id>
<title>This is the title</title>
</data>
I then have some C# code that gets the innerText of title and checks if this value exists in another table. Something like this:
private void TestTitleExist(string innerTextTitle)
{
SqlConnection conn = new SqlConnection("Data Source=;Initial Catalog=;Persist Security Info=True;User ID=;Password=");
conn.Open();
SqlCommand command = new SqlCommand("select 'x' from titles where lower(title) = :#title", conn);
command.Parameters.AddWithValue("#title",innerTextTitle);
using (SqlDataReader reader = command.ExecuteReader())
{
if (reader.Read())
{
Console.WriteLine(String.Format("{0}",reader["title"]));
}
}
conn.Close();
}
After debugging this method, the innerText title is being passed into this method.
For some reason, this query is returning empty even though this title exists in the titles table.
If I run the query directly in SQL server:
select 'x' from titles where title = 'This is the title'
I get 0 rows. But if I remove the white spaces from the title itself (This is the title) and add the spaces back in manually and run the query I get 1 row as expected. It seems like the actual spaces in the titles are adding some kind of formatting. Any idea?
It is better to use XML data type column for any data in XML format.
Please try the following conceptual example for your exact scenario.
It covers just SQL Server side. All you need to do is to call its SQL statement. The idea is very simple:
Convert NTEXT data type into XML data type.
Query XML data by using powerful XQuery methods available in SQL Server.
Casting as xs:token does the following 'magic':
All invisible TAB, Carriage Return, and Line Feed characters will be
replaced with spaces.
Then leading and trailing spaces are removed from the value.
Further, contiguous occurrences of more than one space will be
replaced with a single space.
SQL #1
-- DDL and sample data population, start
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xmldata NTEXT);
INSERT INTO #tbl (xmldata) VALUES
(N'<data>
<id>123</id>
<title>This is the title</title>
</data>'),
(N'<data>
<id>123</id>
<title>This is NOT the title</title>
</data>');
-- DDL and sample data population, end
;WITH rs AS
(
SELECT ID, TRY_CAST(xmldata AS XML) AS xmldata
FROM #tbl
)
SELECT *
FROM rs
WHERE rs.xmldata.exist('/data[title/text()="This is the title"]') = 1;
Output
+----+-----------------------------------------------------------+
| ID | xmldata |
+----+-----------------------------------------------------------+
| 1 | <data><id>123</id><title>This is the title</title></data> |
+----+-----------------------------------------------------------+
SQL #2
Second scenario covers embedded white spaces in the XML.
-- DDL and sample data population, start
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xmldata NTEXT);
INSERT INTO #tbl (xmldata) VALUES
(N'<data>
<id>123</id>
<title>This is the title </title>
</data>'),
(N'<data>
<id>123</id>
<title>This is NOT the title</title>
</data>');
-- DDL and sample data population, end
;WITH rs AS
(
SELECT ID, TRY_CAST(xmldata AS XML) AS xmldata
FROM #tbl
)
SELECT *
FROM rs
WHERE rs.xmldata.exist('/data[string((title/text())[1]) cast as xs:token? eq "This is the title"]') = 1;
Output
+----+----------------------------------------------------------------+
| ID | xmldata |
+----+----------------------------------------------------------------+
| 1 | <data><id>123</id><title>This is the title </title></data> |
+----+----------------------------------------------------------------+
I am trying to pull from a SQL database and produce a formatted XML file.
Looking at previous questions, the best way to do this is using LINQ.
So far, I have the SQL data pulling back into a C# DataSet, there are two tables.
I am using the following code to try and output to an XML doc
XDocument doc = new XDocument(
new XDeclaration("1.0", "UTF-8", "yes"),
new XElement("Snapshot",
new XAttribute("Date", "2018-06-04"),
new XElement("Rooms",
from p in data.Tables[1].AsEnumerable()
select new XElement("RoomType1",
new XAttribute("Rate", p["Rate"]),
new XAttribute("Size", p["Size"]),
new XAttribute("Other", p["Other"]),
)),
new XElement("Cutomers",
from p in data.Tables[2].AsEnumerable()
select new XElement("Customer",
new XAttribute("CustomerId", p["CustomerId"]),
new XAttribute("Name", p["Name"]),
new XAttribute("PhoneNumber", p["PhoneNumber"]),
new XAttribute("Email", p["Email"])
))
doc.Save(#"c:\temp\WholeFile.xml");
This almost produces what I am looking for but on the second data set I want to group by the customerId, so CustomerId will only show once, then the name, phonenumber and email will all be displayed below.
I had a look at grouping using LINQ as it looks as though this is what I need to do but when I run the below code it groups by the customerId correctly but the CustomerId element is still showing on each row of data.
var groups = doc.Descendants("Portfolio")
.GroupBy(x => (string)x.Attribute("PortfolioId"))
.ToList();
var newDoc = new XDocument(
new XElement("Customers",
from e in doc.Descendants("Customer")
group e by (string)e.Attribute("CustomerId") into g
select new XElement("Customer",
new XAttribute("CustomerId", g.Key),
g
)
)
);
Have tried just about everything but can't work out how to pick the fields from the collection (is that the right term) "g".
Any help greatly appreciated!
I personally find it way easier to create the XML on the SQL Server. Here's how I would approach this problem:
USE [yourDB]
GO
/****** Object: StoredProcedure [dbo].[SP_yourProcedure] Script Date: today ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: you
-- Create date: today
-- Description:
-- =============================================
ALTER PROCEDURE [dbo].[SP_yourProcedure]
-- Add the parameters for the stored procedure here
#example1 varchar(50) = '',
#example2 varchar(50) = '',
#example3 varchar(50) = '',
#example4 int = 0
-- ect
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Initialize XML
DECLARE #xml xml = (
SELECT
-- Declare XML Structure
TA.example1 AS 'example1',
TA.example2 AS 'example2',
TA.example3 AS 'example3',
TA.example4 AS 'example4'
-- ect
-- Specify the Context of the XML
FROM yourTable TA WHERE yourConditionFactor = #yourConditionFactor
FOR XML PATH('yourXML'))
-- Do what you want with your #xml here
-- enter code
END
I find it very easy and minimalistic to define the XML structure.
You could then call the stored procedure into your c# environment by using EF and LINQ and just pass the data parameters to your procedure.
If you have multiple XML's you want to create, just let your SP run in a foreach statement.
Hope this helps!
I have a SQL Server 2008 R2 datatable dbo.Forum_Posts with columns Subject (nvarchar(255)) andBody (nvarchar(max)).
I would like to get all words with length >= 3 from columns Subject and Body and insert them into datatable dbo.Search_Word (column Word, nvarchar(100)) and datatable dbo.SearchItem (column Title (nvarchar(200)).
I also want to get new generated SearchWordsID (primary key, autoincrement, int) from dbo.Search_Word, and SearchItemID (primary key, autoincrement,int) from dbo.SearchItem, and insert them into datatable dbo.SearchItemWord (columns SearchWordsID (foreign key,int, not null) and SearchItemID (foreign key,int,not null).
What is a fastest way to do this in T-SQL? Or I have to use C#? Thank you in advance for any help.
As requested, this will keep the ID's. So you will get a DISTINCT list of works BY id.
Slightly different approach than the first answer, but easily achieved via the Outer Apply
**
You must edit the initial query Select KeyID=[YourKeyID],Words=[YourField1]+' '+[YourField2] from [YourTable]
**
Declare #String varchar(max) = ''
Declare #Delimeter varchar(25) = ' '
-- Generate and Strip special characters
Declare #StripChar table (Chr varchar(10));Insert Into #StripChar values ('.'),(','),('/'),('('),(')'),(':') -- Add/Remove as needed
-- Generate Base Data and Expand via Outer Apply
Declare #XML xml
Set #XML = (
Select A.KeyID
,B.Word
From ( Select KeyID=[YourKeyID],Words=[YourField1]+' '+[YourField2] from [YourTable]) A
Outer Apply (
Select Word=split.a.value('.', 'varchar(150)')
From (Select Cast ('<x>' + Replace(A.Words, #Delimeter, '</x><x>')+ '</x>' AS XML) AS Data) AS A
Cross Apply data.nodes ('/x') AS Split(a)
) B
For XML RAW)
-- Convert XML to varchar(max) for Global Search & Replace (could be promoted to Outer Appy)
Select #String = Replace(Replace(cast(#XML as varchar(max)),Chr,' '),' ',' ') From #StripChar
Select #XML = cast(#String as XML)
Select Distinct
KeyID = t.col.value('#KeyID', 'int')
,Word = t.col.value('#Word', 'varchar(150)')
From #XML.nodes('/row') AS t (col)
Where Len(t.col.value('#Word', 'varchar(150)'))>3
Order By 1
Returns
KetID Word
0 UNDEF
0 Undefined
1 HIER
1 System
2 Control
2 UNDEF
3 JOBCONTROL
3 Market
3 Performance
...
87 Analyitics
87 Market
87 UNDEF
88 Branches
88 FDIC
88 UNDEF
...
You're going to need T-SQL to do the inserting into your tables. Your biggest challenge is going to be splitting the posts into words.
My suggestion would be to read the posts into C#, split each post into words (you can use the Split method to split on spaces or punctuation), filter the collection of words, and then execute your inserts from C#.
You can avoid using T-SQL directly if you use Entity Framework or a similar ORM.
Don't try to use T-SQL to split your posts into words unless you really want a totally SQL solution and are willing to take time to perfect it. And, yes, it will be slow: T-SQL is not fast at string operations.
You can also investigate full text indexing, which I believe has support for search keywords.
Perhaps this will help
Declare #String varchar(max) = ''
Declare #Delimeter varchar(25) = ' '
Select #String = #String + ' '+Words
From (
Select Words=[YourField1]+' '+[YourField2] from [YourTable]
) A
-- Generate and Strip special characters
Declare #StripChar table (Chr varchar(10));Insert Into #StripChar values ('.'),(','),('/'),('('),(')'),(':') -- Add/Remove as needed
Select #String = Replace(Replace(#String,Chr,' '),' ',' ') From #StripChar
-- Convert String into XML and Split Delimited String
Declare #Table Table (RowNr int Identity(1,1), String varchar(100))
Declare #XML xml = Cast('<x>' + Replace(#String,#Delimeter,'</x><x>')+'</x>' as XML)
Insert Into #Table Select String.value('.', 'varchar(max)') From #XML.nodes('x') as T(String)
-- Generate Final Resuls
Select Distinct String
From #Table
Where Len(String)>3
Order By 1
Returns (sample)
String
------------------
Access
Active
Adminstrators
Alternate
Analyitics
Applications
Branches
Cappelletti
City
Class
Code
Comments
Contact
Control
Daily
Data
Date
Definition
Deleted
Down
Email
FDIC
Variables
Weekly
Hi I am trying to insert data into an SQL Server Database using an XML file which has some data as follows.I was able to do the attribute mapping in OPENXML.If i try to pass the XML as elements instead of attributes i get an error regarding the null insertion.
Following is my XML FILE (containg attributes)
<NewDataSet>
<SampleDataTable id="20" Name="as" Address="aaa" Email="aa" Mobile="123" />
</NewDataSet>
I am successful using the above format.If i use the below format i face errors
<Customer>
<Id>20</Id>
<Name>Cn</Name>
<Address>Pa</Address>
<Email>bnso#gmail.com</Email>
<Mobile>12345513213</Mobile>
</Customer>
This is my openXML in SQL
insert into #tempTable
select * from openxml (#xmlHandle,'ROOT/Customer/',1)
with (Cust_id int '#id',
Customer_Name varchar(30) '#Name',
Address varchar(30) '#Address',
Email_id varchar(30) '#Email',
Mobile_no bigint '#Mobile'
)
Insert into Test.dbo.tblCustomers (Cust_id,Customer_Name,Address,Email,Mobile_No) (select * from #tempTable)
please help
It's because you're trying to fetch data as attributes, but int your xml data is inside elements. Try this:
insert into #tempTable
select *
from openxml (#xmlHandle,'ROOT/Customer/',1)
with (Cust_id int '#id',
Customer_Name varchar(30) 'Name[1]',
Address varchar(30) 'Address[1]',
Email_id varchar(30) 'Email[1]',
Mobile_no bigint 'Mobile[1]'
)
Or you can do this without openxml:
select
t.c.value('Name[1]', 'varchar(30)') as Name,
t.c.value('Address[1]', 'varchar(30)') as Address,
t.c.value('Email[1]', 'varchar(30)') as Email,
t.c.value('Mobile[1]', 'bigint') as Mobile
from #Data.nodes('Customer') as t(c)
sql fiddle demo
The problem is "element value" vs "element attribute".
This page has good examples of both:
http://technet.microsoft.com/en-us/library/ms187897%28v=sql.90%29.aspx
<Customer>
<CustomerID>LILAS</CustomerID>
<ContactName>Carlos Gonzlez</ContactName>
<Order OrderID="10283" CustomerID="LILAS" EmployeeID="3" OrderDate="1996-08-16T00:00:00">
<OrderDetail ProductID="72" Quantity="3"/>
</Order>
</Customer>
</ROOT>'
-- Create an internal representation of the XML document.
EXEC sp_xml_preparedocument #XmlDocumentHandle OUTPUT, #XmlDocument
-- Execute a SELECT statement using OPENXML rowset provider.
SELECT *
FROM OPENXML (#XmlDocumentHandle, '/ROOT/Customer',2)
WITH (CustomerID varchar(10),
ContactName varchar(20))
EXEC sp_xml_removedocument #XmlDocumentHandle
I'm attempting to create a small console app in C# to perform inserts on a table of Products (ITEMS) in SQL Server 2008 according to the contents of an XML file in the FASTEST way possible. I already have an .XSD file that contains the proper mappings to the SQL table (which may not be necessary with the approach outlined below).
Here's a high-level of my approach:
Read the XML, using it to create a table.
Perform a MERGE against the ITEMS table using the table created from the XML file.
2a. If the item exists, update it.
2b. If the item does not exist, insert it.
Create a log of only the records inserted in XML.
Consider the following ITEMS table and XML file:
ITEMS
Item_Id Name Price
1 Coke 5.00
2 Pepsi 3.00
3 Sprite 2.00
ITEMS.XML
<?xml version="1.0" encoding="ISO-8859-1"?>
<Item>
<Id>5</Id>
<Name>Mountain Dew</Name>
<Price>4.50</Price>
</Item>
<Item>
<Id>3</Id>
<Name>Sprite Zero</Name>
<Price>1.75</Price>
</Item>
After the import, the ITEMS table should look like:
ITEMS
Item_Id Name Price
1 Coke 5.00
2 Pepsi 3.00
3 Sprite Zero 1.75
5 Mountain Dew 4.50
Once that's done, I also need to generate an XML formatted log file that contains the "new" record that was inserted into the table (ITEMS_LOG.XML):
ITEMS_LOG.XML
<?xml version="1.0" encoding="ISO-8859-1"?>
<Item>
<Id>5</Id>
<Name>Mountain Dew</Name>
<Price>4.50</Price>
</Item>
I have tried implementing this using SQLXMLBulkLoad, but unfortunately it does not provide the logging that I need, nor does it permit me to access any of the messages returned from SQL Server (i.e. what's been inserted/updated). Although I have an intermediate level of SQL expertise, I am fairly new to working with XML, especially in this context. Any help/guidance would be greatly appreciated!
You can use merge with output to a table variable and then query the table variable to build the log XML.
Put it in a stored procedure where you have the item XML as an in parameter and the log XML as an out parameter.
create procedure AddItemXML
#ItemsXML xml,
#ItemsLogXML xml out
as
declare #Changes table
(
Item_Id int,
Name nvarchar(20),
Price money,
Action nvarchar(10)
);
merge Items as T
using
(
select T.N.value('Id[1]', 'int') as Item_Id,
T.N.value('Name[1]', 'varchar(20)') as Name,
T.N.value('Price[1]', 'money') as Price
from #ItemsXML.nodes('/Item') T(N)
) as S
on T.Item_Id = S.Item_Id
when matched then
update set Name = S.Name, Price = S.Price
when not matched then
insert (Item_Id, Name, Price) values (S.Item_Id, S.Name, S.Price)
output inserted.Item_Id,
inserted.Name,
inserted.Price,
$action
into #Changes;
set #ItemsLogXML =
(
select Item_Id as ID,
Name,
Price
from #Changes
where Action = 'INSERT'
for xml path('Item'), type
);
Working sample on SE-Data
Hope this helps you, What I did was to create a stored procedure as below. Basically the stored procedure takes xml values and checks the flags which are passed from code and determines if it is insert or update:
DECLARE #xml xml
SET #xml = #xmlCredentials
SELECT
item.value('#Id', 'int') As ID,
item.value('#AgentID', 'int') As AgentID,
item.value('#Username', 'varchar (50)') As Username,
item.value('#Password', 'varchar (50)') As [Password],
item.value('#IsDirty', 'bit') As IsDirty,
item.value('#IsDeleted', 'bit') As IsDeleted
INTO #tmp
FROM #xml.nodes('Credentials/Credential') x(item)
BEGIN TRY
BEGIN TRAN
INSERT INTO Credentials (AgentID, Username, [Password])
SELECT
AgentID, Username, [Password]
FROM
#tmp
WHERE
ID = 0 AND IsDirty = 1
UPDATE c
SET c.[AgentID] = t.AgentID,
c.[Username] = t.Username,
c.[Password] = t.[Password]
FROM
[dbo].[Credentials] c
JOIN
#tmp t ON t.Id = c.ID
WHERE
t.IsDirty = 1 AND t.IsDeleted = 0
DELETE FROM [dbo].[Credentials]
FROM [dbo].[Credentials] c
JOIN #tmp t ON t.Id = c.ID
WHERE
t.IsDirty = 1 AND t.IsDeleted = 1
COMMIT TRAN
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
ROLLBACK TRAN
DECLARE #errorMSG varchar(4000)
DECLARE #errorSeverity int
DECLARE #errorState int
SET #errorMSG = ERROR_MESSAGE()
SET #errorSeverity = ERROR_SEVERITY()
SET #errorState = ERROR_STATE()
RAISERROR (#errorMSG,
#errorSeverity, #errorState);
END CATCH
SELECT [ID], [AgentID], [Username], [Password]
FROM [dbo].[Credentials]
In code behind I have my xml and pass the xml as parameter to the stored procedure:
// read xml and assign it to string variable
string xml = readxml();
try
{
string command = "EXEC SaveCredentails '" + xml + "'";
}
catch(Exception e)
{
}
I would use a staging table to import the xml into a SQL Server table. Add an extra column to indicate the action (insert or update). Then use regular sql to perform upserts as usual. You can then use the staging table to generate the XML logging you need (reading the action column to determine if it was an insert or update).