I've got a stored procedure which joins a number of tables to produce a large resultset which is then returned to my application. The application in turn loops through the results and combines rows on a particular ID and chooses data per row to include in a new object. This is perhaps easiest to explain using an example:
Inspection, Desc, Value
1, Description1, 3
1, Description2, 2
1, Description3, 5
This is in code turned into
Inspection, Description1, Description2, Description3
1, 3, 2, 5
The point of this is to have one row per inspection item with item description as headers and value as the cell value for inspection row and header. This is then exported to Excel.
The question is: how do I do this in SQL Server, as in expanding my SP to return a lot fewer but "wider" rows with a lot more columns?
Another complication is that one inspection may have rows which another one lacks, in that case the solution is to add an empty value or a '-'.
P.S. This is using Sql Server 2012.
If you are using mssql 2005+. You can use a pivot like this:
Test data
DECLARE #tbl TABLE(Inspection INT, [Desc] VARCHAR(100),Value INT)
INSERT INTO #tbl
VALUES
(1,'Description1', 3),
(1,'Description2', 2),
(1,'Description3', 5)
Query
SELECT
*
FROM
(
SELECT
tbl.Inspection,
tbl.[Desc],
tbl.Value
FROM
#tbl AS tbl
) AS tbl
PIVOT
(
SUM(Value)
FOR [Desc] IN ([Description1],[Description2],[Description3])
)AS pvt
Result:
Inspection, Description1, Description2, Description3
1 3 2 5
Edit
As juharr said in the comment:
The resulting column names (values in the table) are when building the query. Which might require another initial query to get
Edit 2
If you are not using mssql 2005+. Or want to have and alternitive explanation. Please see the following query:
SELECT
tbl.Inspection,
SUM(CASE WHEN [Desc]='Description1' THEN tbl.Value ELSE 0 END) AS Description1,
SUM(CASE WHEN [Desc]='Description2' THEN tbl.Value ELSE 0 END) AS Description2,
SUM(CASE WHEN [Desc]='Description3' THEN tbl.Value ELSE 0 END) AS Description3
FROM
#tbl AS tbl
GROUP BY
tbl.Inspection
This do not requiere a pivot and can be use on most of RDMS out there
You should use Sql Server Pivot. It converts rows into columns. You can have an easiest start by this example.
If you'd like to do this dynamically, without having to know what all of the Desc values are, you can build your pivot query and use Exec() or Execute sp_executesql
DECLARE #Columns NVARCHAR(MAX),
#Sql NVARCHAR(MAX)
--Build your column headers based on Distinct Desc values
SELECT #Columns = COALESCE(#Columns + ',', '') + QUOTENAME([Desc])
FROM (SELECT DISTINCT [Desc] FROM tbl) t
ORDER BY [Desc]
--Build your pivot query
SET #Sql = '
SELECT
*
FROM
tbl
PIVOT
(
MAX([Value])
FOR [Desc] IN (' + #Columns + ')
) p
'
EXEC(#Sql)
If you want - for null values, you'll need to create another variable to hold the conversion scripts for the Select part of your sql.
DECLARE #Columns NVARCHAR(MAX),
#Sql NVARCHAR(MAX),
#ColumnAliases NVARCHAR(MAX)
--Build your pivot columns based on Distinct Desc values
SELECT #Columns = COALESCE(#Columns + ',', '') + QUOTENAME([Desc])
FROM (SELECT DISTINCT [Desc] FROM tbl) t
ORDER BY [Desc]
--Build your column headers, replacing NULL with -
SELECT #ColumnAliases = COALESCE(#ColumnAliases + ',', '')
+ 'COALESCE(CONVERT(VARCHAR,' + QUOTENAME([Desc]) + '),''-'') AS ' + QUOTENAME([Desc])
FROM (SELECT DISTINCT [Desc] FROM tbl) t
ORDER BY [Desc]
--Build your pivot query
SET #Sql = '
SELECT
Inspection,'
+ #ColumnAliases + '
FROM
tbl
PIVOT
(
MAX([Value])
FOR [Desc] IN (' + #Columns + ')
) p
'
EXEC(#Sql)
Related
I want to duplicate all my tables in SQL Server, all table names would have had "temp" added at the beginning. And all of them would have had added an extra column (the same to all). I don't need whole code, just general idea how to do it.
A straightforward way to go:
You need to fetch the table names from your database (probably using INFORMATION_SCHEMA.TABLES).
For each of those tables from step 1, you need to generate a corresponding SELECT ... INTO statement.
You need to execute each generated SQL statement from step 2.
You already have a solution with a cursor. This is one without a cursor:
DECLARE #script VARCHAR(MAX) = '';
SELECT #script = #script + 'SELECT * INTO [temp'+ TABLE_NAME +'] FROM [' + TABLE_NAME + '];' + CHAR(13) + CHAR(10) FROM INFORMATION_SCHEMA.TABLES
EXEC (#script);
Remark: The CHAR(13) + CHAR(10) is not necessary; just added for readability if you want to check the script first (using PRINT instead EXEC).
Edit:
An additional question in the comments to add a checksum value in the resulting tables could be done as follows:
DECLARE #script VARCHAR(MAX) = '';
SELECT #script = #script + 'SELECT CHECKSUM(*) AS [__checksum], * INTO [temp'+ TABLE_NAME +'] FROM [' + TABLE_NAME + '];' + CHAR(13) + CHAR(10) FROM INFORMATION_SCHEMA.TABLES
EXEC (#script);
Using HASHBYTES instead of CHECKSUM is probably better, but it accepts only two parameters: the hash algorithm and a single value to hash. So in that case, you probably need to pass a string value by manually concatenating all the fields of your tables, and that may be somewhat troublesome to add in a dynamic query like mine. It would probably result in something more complex than just three lines...
Well, something like this, actually:
DECLARE #script NVARCHAR(MAX) = N'';
WITH
[Columns] AS
(
SELECT
TABLE_NAME AS [TableName],
COLUMN_NAME AS [ColumnName],
ROW_NUMBER() OVER (PARTITION BY TABLE_NAME ORDER BY ORDINAL_POSITION) AS [ColSeq]
FROM
INFORMATION_SCHEMA.COLUMNS
),
[Tables] AS
(
SELECT
[TableName],
CAST(N'[' + [ColumnName] + N']' AS NVARCHAR(MAX)) AS [ColumnList],
[ColSeq]
FROM
[Columns] AS C
WHERE
[ColSeq] = (SELECT MAX([ColSeq])
FROM [Columns]
WHERE [TableName] = C.[TableName])
UNION ALL
SELECT T.[TableName], N'[' + C.[ColumnName] + N'], ' + T.[ColumnList], C.[ColSeq]
FROM
[Tables] AS T
INNER JOIN [Columns] AS C ON C.[TableName] = T.[TableName] AND C.[ColSeq] = T.[ColSeq] - 1
)
SELECT #script = #script + N'SELECT HASHBYTES(''md5'', CONCAT(N'''', ' + [ColumnList] + N')) AS [__checksum], * INTO [temp' + [TableName] + N'] FROM [' + [TableName] + N'];' + NCHAR(13) + NCHAR(10)
FROM [Tables]
WHERE [ColSeq] = 1;
EXEC (#script);
Remarks:
In the recursive CTE [Tables], which is used for concatenating the column names of each table in a comma-separated string value, I started at the last column and moved backwards to ease the filter condition in my main query.
I added an additional first parameter N'' to the CONCAT calls in the resulting #script contents, since the CONCAT function requires at least 2 arguments, which would be troublesome in this case when processing tables with just one column.
In this case, despite the somewhat worse performance, it might be clearer and easier to fall back to using a cursor, like #HasanMahmood suggested in his answer...
try this code:
get all the table name form information schema and run a dynamic sql to create tables
DECLARE #script varchar(max)
DECLARE db_cursor CURSOR FOR
SELECT script = 'Select * Into [temp'+ TABLE_NAME +'] From ' + QUOTENAME(TABLE_NAME) FROM INFORMATION_SCHEMA.TABLES
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO #script
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC(#script)
--PRINT #script
FETCH NEXT FROM db_cursor INTO #script
END
CLOSE db_cursor
DEALLOCATE db_cursor
I would like to transpose the data from my table and do some plottings into powerBI.
Here is how I feel up my database from my application:
using (SqlCommand cmd = connect.CreateCommand())
{
cmd.CommandText = #"INSERT INTO PoD_NewPriceList_Data
(ID, Product_Barcode, Product_Name,
Store_Price, Internet_Price, InsertDate)
VALUES (#ID, #Product_Barcode, #Product_Name,
#Store_Price, #Internet_Price, #InsertDate)";
cmd.Parameters.Add("Product_Barcode", SqlDbType.NVarChar).Value = barcode;
cmd.Parameters.Add("Product_Name", SqlDbType.NVarChar).Value = PriceList.name;
cmd.Parameters.Add("Store_Price", SqlDbType.Float).Value = Convert.ToDouble(storePrice, CultureInfo.InvariantCulture);
cmd.Parameters.Add("Internet_Price", SqlDbType.Float).Value = Convert.ToDouble(PriceList.price, CultureInfo.InvariantCulture);
cmd.Parameters.Add("InsertDate", SqlDbType.DateTime).Value = InsertDate.AddDays(2);
cmd.Parameters.Add("ID", SqlDbType.Int).Value = barcode.GetHashCode();
result = result && (cmd.ExecuteNonQuery() > 0);
}
And in SQL Server Management Studio here is how my table looks like:
SELECT
[ID], [Product_Barcode], [Product_Name],
[Store_Price], [Internet_Price], [InsertDate]
FROM
[dbo].[PoD_NewPriceList_Data]
and I get the following output:
The main issue is when trying to create the plots as requested in PowerBI I need my data to look as follows:
F5321
Product_Name Sony Xperia...
Store_Price 399
Internet_Price 327.51
InsertDate 2017.04.27
Any help would be well appreciated.
Check and modify this SQL script. I use #t table variable, replace it with your table name [PoD_NewPriceList_Data].
DECLARE #t TABLE (
id int,
product_barcode varchar(max),
product_name varchar(max),
store_price int,
internet_price decimal,
insert_date date
)
INSERT INTO #t VALUES (1,'F5321', 'Sony Xperia', 399, 255.1, '2017-04-25')
INSERT INTO #t VALUES (2,'F5833', 'Sony Xperia XZ', 458, 398.2, '2017-04-26')
INSERT INTO #t VALUES (3,'F5121', 'Sony Xperia XA Rose', 161, 155.6, '2017-04-27')
IF OBJECT_ID ('tempdb..#Unpivoted') IS NOT NULL
DROP TABLE #Unpivoted
IF OBJECT_ID ('tempdb..#Transposed') IS NOT NULL
DROP TABLE #Transposed
/* Unpivot table to get rows instead of columns */
SELECT *, ROW_NUMBER() OVER (ORDER BY (SELECT 0)) as rn
INTO #Unpivoted
FROM (SELECT product_barcode, product_name,
CAST(store_price as varchar(max)) store_price,
CAST(internet_price as varchar(max)) internet_price,
CAST(insert_date as varchar(max)) as insert_date
FROM #t) src
UNPIVOT (
value FOR field IN (
product_barcode, product_name, store_price, internet_price, insert_date
)
) unpiv
CREATE TABLE #Transposed
(Field varchar(50) PRIMARY KEY NOT NULL )
DECLARE #SQL NVARCHAR(MAX)
SELECT #SQL = STUFF((
SELECT 'ALTER TABLE #Transposed ADD item' +
RIGHT('000' + CAST(sv.number AS VARCHAR(3)), 3) + ' varchar(max) '
FROM [master].dbo.spt_values sv
WHERE sv.[type] = 'p'
AND sv.number BETWEEN 1 AND (SELECT COUNT(*) FROM #t)
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 0, '')
Exec(#SQL) /* Dynamically create columns */
INSERT INTO #Transposed (Field) SELECT DISTINCT Field FROM #Unpivoted
/*populate field names*/
DECLARE #fieldCount int = (SELECT COUNT(*) FROM #Transposed)
/* using rn to filter proper record from transposed table */
SELECT #SQL = STUFF((
SELECT '
UPDATE #Transposed SET item' + RIGHT('000' + CAST(sv.number AS VARCHAR(3)), 3)
+ ' = up.value FROM #Transposed t CROSS APPLY
( SELECT TOP 1 u.value FROM #unpivoted u WHERE u.field = t.field AND u.rn > '
+ CAST((sv.number-1)*#fieldCount AS VARCHAR(10)) + ' ORDER BY rn) up '
FROM [master].dbo.spt_values sv
WHERE sv.[type] = 'p'
AND sv.number BETWEEN 1 AND (SELECT COUNT(*) FROM #t)
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 0, '')
Exec(#SQL) /*Dynamically fill in values */
SELECT t.* FROM #Transposed t
OUTER APPLY (SELECT TOP 1 rn FROM #Unpivoted u WHERE u.field=t.field) up
ORDER BY up.rn ASC /* add a link to Unpivoted to fix the item order */
DROP TABLE #Unpivoted
DROP TABLE #Transposed
It does what you need in several steps
converts columns to rows with UNPIVOT. Watch that you have to CAST all the values to the exactly same type. Adds a row number to filter the rows in step 3.
creates a temp table with dynamic number of columns corresponding to the number of rows
fills in the columns names into rows into the dynamically created table
fills in values into the dynamically created table
Credits to this answer and this answer.
Of course the number of columns is limited here, so if you try to convert many rows into columns, you get:
Cannot create a row of size 8066 which is greater than the allowable
maximum row size of 8060.
Does anyone have a script to list of CREATE INDEX statements for all existing indexes in a SQL Server database?
This thread List of all index & index columns in SQL Server DB has great tips on how to find them. But a script to generate the CREATE INDEX statements would be great. Sometimes we come into a situation without adequate data, or indexes have been added in an ad-hoc manner over time without documentation, so the create statements are missing. Like in a situation I find myself in right now.
Thanks.
Use Generate Scripts from SQL Management Studio and choose the "Script Indexes" options (under Advanced Scripting options)
I wrote something for that a while ago. You might have to modify it for your needs, but at least you have a skeleton.
if exists (select 1 from information_schema.routines where routine_name = 'Script_CreateIndex')
drop proc Script_CreateIndex
go
create proc Script_CreateIndex (
#TableName varchar(124)
)
as
begin
if not exists (select 1 from sys.indexes where object_name(object_id) = #TableName and type_desc in ('CLUSTERED', 'NONCLUSTERED'))
return
declare #IndexList table (
Id int identity,
IndexName varchar(124),
IndexDescription varchar(max),
IndexKeys varchar(max)
)
insert #IndexList(IndexName, IndexDescription, IndexKeys)
exec sp_helpindex #TableName
if (select count(*) from #IndexList) > 0
begin
select '-- Creating indexes for table ' + #TableName
while exists (select 1 from #IndexList)
begin
declare #Id int, #IndexName varchar(124), #IndexDescription varchar(max), #IndexKeys varchar(max)
select top 1 #Id = Id, #IndexName = IndexName, #IndexDescription = IndexDescription, #IndexKeys = IndexKeys from #IndexList order by Id
delete from #IndexList where Id = #Id
declare #Clustered varchar(10), #Unique varchar(7)
select #Clustered = case when patindex('%nonclustered%', #IndexDescription) > 0 then '' else ' clustered ' end
select #Unique = case when patindex('%unique%', #IndexDescription) > 0 then ' unique ' else '' end
select 'if not exists (select 1 from sys.indexes where name = ''' + #IndexName + ''')'
select 'begin'
select char(9) + 'create' + #Unique + #Clustered + ' index [' + #IndexName + '] on [' + #TableName + '](' + #IndexKeys + ')'
select char(9) + 'select ''Index ' + #IndexName + ' created.'''
select 'end'
select 'go'
end
select ''
select ''
end
end
go
grant exec on Script_CreateIndex to public
select 'Script_CreateIndex compiled.' 'Job'
go
Check my solution here:
https://stackoverflow.com/a/55742250/1831734
Output
Create Drop Rebuild
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CREATE CLUSTERED INDEX [PK_Table1] ON [Table1] ( [Tab1_ID] ) DROP INDEX [PK_Table1] ON [Table1] ALTER INDEX [PK_Table1] ON [Table1] REBUILD
CREATE UNIQUE INDEX [IX_Table1_Name] ON [Table1] ( [Tab1_Name] ) DROP INDEX [IX_Table1_Name] ON [Table1] ALTER INDEX [IX_Table1_Name] ON [Table1] REBUILD
CREATE NONCLUSTERED INDEX [IX_Table2] ON [Table2] ( [Tab2_Name], [Tab2_City] ) INCLUDE ( [Tab2_PhoneNo] ) DROP INDEX [IX_Table2] ON [Table2] ALTER INDEX [IX_Table2] ON [Table2] REBUILD
You can do it on a table by table basis by using the "Object Explorer" window
Go to the Indexes folder in Management studio, highlight the folder then open the Object Explorer pane
You can then "shift Select" all of the indexes on that table, if you right click to script "CREATE TO" it will create a script with all the relevant indexes for you.
I'm working on a webapp that displays data from a Microsoft SQL Server dynamic pivot table.
Normally I'd try and figure out a way to do the dynamic pivot in c#, but in this case the pivot has to be a SQL Server stored procedure because other apps also need access to the pivot table.
Here's the SQL:
DECLARE #DynamicPivot AS NVARCHAR(MAX)
DECLARE #ColumnName AS NVARCHAR(MAX)
SELECT
#ColumnName = ISNULL(#ColumnName + ',', '')
+ QUOTENAME(xml_tag_name)
FROM
(SELECT DISTINCT xml_tag_name FROM DataEntries) AS TagValues
SET #DynamicPivot =
N'SELECT DISTINCT capture_id, ' + #ColumnName + '
FROM DataEntries
PIVOT(MAX(xml_tag_value)
FOR xml_tag_name IN (' + #ColumnName + ')) AS PVTTable'
EXEC sp_executesql #DynamicPivot
All the articles I've gone through deal with normal export to Excel or static pivots. eg: Export Table from SQL Server to Excel 2007 using C#.
How do I go about exporting this dynamic pivot to Excel?
heres a example for you.
It creates dynamic pivot to global temp table and then exports it to excel.
If you find problems with export part - let me know.
CREATE TABLE Table1 (ColId INT,ColName VARCHAR(10))
INSERT INTO Table1 VALUES(1, 'Country')
INSERT INTO Table1 VALUES(2, 'Month')
INSERT INTO Table1 VALUES(3, 'Day')
CREATE TABLE Table2 (tID INT,ColID INT,Txt VARCHAR(10))
INSERT INTO Table2 VALUES (1,1, 'US')
INSERT INTO Table2 VALUES (1,2, 'July')
INSERT INTO Table2 VALUES (1,3, '4')
INSERT INTO Table2 VALUES (2,1, 'US')
INSERT INTO Table2 VALUES (2,2, 'Sep')
INSERT INTO Table2 VALUES (2,3, '11')
INSERT INTO Table2 VALUES (3,1, 'US')
INSERT INTO Table2 VALUES (3,2, 'Dec')
INSERT INTO Table2 VALUES (3,3, '25')
DECLARE #cols NVARCHAR(2000);
SELECT #cols = COALESCE(#cols + ',[' + colName + ']', '[' + colName + ']')
FROM Table1
ORDER BY colName;
IF OBJECT_ID('tempdb..##t1') IS NOT NULL
BEGIN
DROP TABLE ##t1;
END;
DECLARE #query NVARCHAR(4000);
SET #query = N'SELECT tID, ' + #cols + ' into ##t1
FROM
(SELECT t2.tID
, t1.ColName
, t2.Txt
FROM Table1 AS t1
JOIN Table2 AS t2 ON t1.ColId = t2.ColID) p
PIVOT
(
MAX([Txt])
FOR ColName IN
( ' + #cols + ' )
) AS pvt
ORDER BY tID;';
EXECUTE(#query);
SELECT *
FROM ##t1;
DECLARE #sql VARCHAR(MAX);
DECLARE #FileName VARCHAR(MAX) = 'C:\Test.xls';
SET #sql = 'INSERT INTO OPENROWSET(''Microsoft.ACE.OLEDB.12.0'',''Excel 12.0;Database='
+ #FileName + ''',''SELECT * FROM [Sheet1$]'') SELECT * FROM ##T1';
EXECUTE(#sql);
I have a int array of ID's (a lot of checkboxes I can choose from) which I want to get in one database call though a stored procedure.
Is there a way to work with an array of these ID's in SQL Server? I believe it should be something with splitting the array and then loop it (in sql). I just don't know how?
SQL Server 2008
There are many ways to do this:
Pass in a varchar parameter of the values separated by commas and parse that out (not very efficient, but for a small amount of data, not too bad except for the parsing bit)
Pass in XML and use the built in XML functions (SQL Server 2005+ has better support for this than earlier versions)
Use table value parameters (SQL Server 2008+)
Since you are using SQL Server 2008, use table value parameters.
EDIT: Example below
As #Oded mentioned, table valued parameters is the best option.
However, if for some reason you can't use these (perhaps your calling framework's limitations), you can use the following to perform the split to table:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER FUNCTION [dbo].[SplitToTable]
(
#List varchar(max), #Delim varchar(1)
)
RETURNS TABLE
AS
RETURN
(
WITH csvtbl(Start, [Stop]) AS (
SELECT Start = convert(bigint, 1), [Stop] =
charindex(#Delim COLLATE Slovenian_BIN2, #list + #Delim)
UNION ALL
SELECT Start = [Stop] + 1, [Stop] = charindex(#Delim
COLLATE Slovenian_BIN2, #list + #Delim, [Stop] + 1)
FROM csvtbl
WHERE ([Stop] > 0)
)
SELECT substring(#list, Start, CASE WHEN [Stop] > 0 THEN [Stop] -
Start ELSE 0 END) AS Value
FROM csvtbl
WHERE ([Stop] > 0)
)
You need to be aware of the default recursion depth of 100. If this isn't enough, increase it by adding the following to your outer calling query:
OPTION (MAXRECURSION 1000) -- or 0 for unlimited
EXAMPLE
SELECT *
FROM MyTable as t
WHERE t.ID IN (
SELECT *
FROM dbo.SplitToTable('1,2,12,34,101', ',')
)
It can be used on joins, etc., too.
I think you need something like...
Declare #query as varchar(500)
Declare #valuesList as varchar(100)
set #valuesList = '1,2,3'
set #query = 'select * From tableName where id in ( ' + #valuesList + ')'
exec(#query)
TO REVERSE THE PROCESS
DECLARE #t TABLE
(
ID int
)
INSERT INTO #t
VALUES (1), (3), (5), (7), (9)
SELECT STUFF(
(
SELECT ',' + CAST(t.ID AS VARCHAR(10))
FROM #t t
FOR XML PATH('')
), 1, 1, '') AS CSV
Courtesy SQLAuthority.