Related
I have a quick question that why my SQL Stored Procedure did not work properly. Can someone explain what is wrong with my Stored Procedure Query?
Error: "Each GROUP BY expression must contain at least one column that is not an outer reference."
SELECT
#TabGroupBy + #TabGroupByName + ','
,SUM(Value) AS Sum
,[Unit]
,[Child_Name]
FROM (
SELECT [model_id]
,[Child_ID]
,[Property_ID]
,[DDate]
,[Hour]
,[Value]
FROM [RP_IRP].[M_PLEXOS].[dat_Generators]
where parent_ID=1 and child_ID in(9, 357,358) and Property_ID in (4,31)
) a
inner join [RP_IRP].[M_PLEXOS].[Child_Object] b on a.child_id=b.child_id
inner join [M_PLEXOS].[Property] d on d.[Property_ID] = a.[Property_ID]
inner join [M_PLEXOS].[Units] e on d.[Unit_ID]=e.[Unit_ID]
inner join [M_PLEXOS].[Model_Config] f on a.[Model_id]=f.[Model_id]
WHERE Child_Name = #SelectedChildValue AND Property = #SelectedPropertyValue
AND Unit = #SelectedUnitValue
GROUP BY Child_Name , #TabGroupBy , Unit HAVING SUM(Value) > #MinValue
It can never work the way you are trying it, because by definition, all column names in the SELECT-part, as well as all column names in the GROUP BY-clause can not come from #-variables. They must be written as plain text because they are identifiers.
The same applies to table names + column names in FROM and JOIN clauses.
The reason for this is that the query compiler is built to check all specified columns (and tables, and schemas, and more) against objects that exist in your database(s), and this needs to succeed before a single line of compiled code runs. You should always keep in mind that at compilation time, #-variables don't yet exist and can't have values (because they don't yet exist).
The solution is to use dynamic SQL. You can achieve what you want by building the actual SQL string that you want to execute in a #SQL variable of type NVARCHAR(max), and then EXEC the contents of that variable. EXEC will invoke the SQL query compiler on the contents of its parameter.
Example code, may not be 100% perfect because I can't run it due to not having your database available, but this should get you on your way:
DECLARE #SQL NVARCHAR(max) =
'SELECT ' +
QUOTENAME(#TabGroupBy) + ' AS ' + QUOTENAME(#TabGroupByName) + ', ' +
'SUM(Value) AS Sum, ' +
'[Unit], ' +
'[Child_Name] ' +
'FROM ( ' +
'SELECT [model_id],[Child_ID],[Property_ID],[DDate],[Hour],[Value] ' +
'FROM [RP_IRP].[M_PLEXOS].[dat_Generators] ' +
'where parent_ID = 1 ' +
' and child_ID in (9, 357, 358) ' +
' and Property_ID in (4, 31) ' +
') a ' +
'inner join [RP_IRP].[M_PLEXOS].[Child_Object] b on a.child_id=b.child_id ' +
'inner join [M_PLEXOS].[Property] d on d.[Property_ID] = a.[Property_ID] ' +
'inner join [M_PLEXOS].[Units] e on d.[Unit_ID]=e.[Unit_ID] ' +
'inner join [M_PLEXOS].[Model_Config] f on a.[Model_id]=f.[Model_id] ' +
'WHERE Child_Name = ''' + REPLACE(#SelectedChildValue , '''', '''''') + ''' ' +
' AND Property = ''' + REPLACE(#SelectedPropertyValue, '''', '''''') + ''' ' +
' AND Unit = ''' + REPLACE(#SelectedUnitValue , '''', '''''') + ''' ' +
'GROUP BY Child_Name , ' + QUOTENAME(#TabGroupBy) + ', Unit ' +
'HAVING SUM(Value) > ' + CAST(#MinValue AS VARCHAR(20)) -- assuming #MinValue is INT or FLOAT
EXEC (#SQL)
This code assumes there may be quotes inside the #-variables. Always use REPLACE to double embedded quotes if the #-variables represent string values, or even better: use dynamic SQL along with #-parameters, see this Q & A for how that can be done.
For cases where the #-variables represent database identifiers (column names etc.), you need to use QUOTENAME(...) as in the example code to make sure that no abuse can take place.
You cannot parameterize a GROUP BY column. Remove the parameter #TabGroupBy from the GROUP BY
I have a DataSet like below :
Where B1,B2 C1,C2 and C3 are the Column names. G1,G2,S1 and T1 are the first Row elements of my Data Set.
Now i want to combine Similar Columns/ROws in to Group .
Example : Columns B1, B2 and B3 combines to a single group B,
Rows : G1 and G2 Combines together to form a single row G.
Below is the O/P DataSet i need.
I have tried to use Dictionaries and DataSet Loops using but cant get this O/P .
Can anyone help me out with this.
This having to be dynamic adds a huge amount of complexity to this solution. As you haven't responded to the version question, I have not used STRING_AGG, however, if you are using SQL Server 2017+ you can simplify the query to use it.
Firstly, some sample data:
CREATE TABLE dbo.Matrix ([Data] char(2),
B1 tinyint,
B2 tinyint,
C1 tinyint,
C2 tinyint,
C3 tinyint)
INSERT INTO dbo.Matrix ([Data],
B1,
B2,
C1,
C2,
C3)
VALUES('G1',1,1,2,2,4),
('G2',1,1,1,1,1),
('S1',2,1,2,1,1),
('T1',1,3,2,2,3);
GO
Now, if this wasn't dynamic, you could use a a Cross tab to pivot the data into groups, like this:
SELECT LEFT(M.[Data],1) AS [Data],
SUM(CASE V.Col WHEN 'B' THEN V.ColVal END) AS B,
SUM(CASE V.Col WHEN 'C' THEN V.ColVal END) AS C
FROM dbo.Matrix M
CROSS APPLY(VALUES('B',M.B1),
('B',M.B2),
('C',M.C1),
('C',M.C2),
('C',M.C3))V(Col,ColVal)
GROUP BY LEFT(M.[Data],1);
Unfortunately, as it is dynamic then we need dynamic SQL. Honestly, this isn't beginning stuff, and I'm not here to support this SQL; it's up to you to understand it, maintain it, support it, and (because it is dynamic SQL) keep it secure. I'm happy to answer some questions on how it works, but for someone that doesn't knowe SQL well this is a steep learning curve:
DECLARE #SQL nvarchar(MAX);
SET #SQL = N'SELECT LEFT(M.[Data],1) AS [Data],' + NCHAR(13) + NCHAR(10) +
STUFF((SELECT N',' + NCHAR(13) + NCHAR(10) +
N' SUM(CASE V.Col WHEN N' + QUOTENAME(LEFT(C.COLUMN_NAME,1),'''') + N' THEN V.ColVal END) AS ' + QUOTENAME(LEFT(C.COLUMN_NAME,1))
FROM INFORMATION_SCHEMA.COLUMNS C
WHERE C.TABLE_SCHEMA = N'dbo'
AND C.TABLE_NAME = N'Matrix'
AND C.COLUMN_NAME != N'Data' --Assumes that all other columns are applicable
GROUP BY LEFT(C.COLUMN_NAME,1)
ORDER BY LEFT(C.COLUMN_NAME,1)
FOR XML PATH(N''),TYPE).value('.','nvarchar(MAX)'),1,3,N'') + NCHAR(13) + NCHAR(10) +
N'FROM dbo.Matrix M' + NCHAR(13) + NCHAR(10) +
N' CROSS APPLY(VALUES' + STUFF((SELECT ',' + NCHAR(13) + NCHAR(10) +
N' (N' + QUOTENAME(LEFT(C.COLUMN_NAME,1),'''') + N',M.' + QUOTENAME(C.COLUMN_NAME) + N')'
FROM INFORMATION_SCHEMA.COLUMNS C
WHERE C.TABLE_SCHEMA = N'dbo'
AND C.TABLE_NAME = N'Matrix'
AND C.COLUMN_NAME != N'Data' --Assumes that all other columns are applicable
ORDER BY C.COLUMN_NAME
FOR XML PATH(N''),TYPE).value('.','nvarchar(MAX)'),1,26,N'') + N')V(Col,ColVal)' + NCHAR(13) + NCHAR(10) +
N'GROUP BY LEFT(M.[Data],1)' + NCHAR(13) + NCHAR(10) +
N'ORDER BY LEFT(M.[Data],1);';
PRINT #SQL; --Your debugging best friend.
EXEC sp_executesql #SQL;
db<>fiddle
You can apply below logic to get desired output from SQL Server:
create table T_DataSet
(
Data varchar(50),
B1 int,
B2 int,
C1 int,
C2 int,
C3 int
)
INSERT INTO T_DataSet ([Data],B1,B2,C1,C2,C3)
VALUES('G1',1,1,2,2,4),
('G2',1,1,1,1,1),
('S1',2,1,2,1,1),
('T1',1,3,2,2,3);
DECLARE #QUERY VARCHAR(MAX)=''
DECLARE #Columns VARCHAR(MAX)=''
;with tbl_COLUMN_NAME (COLUMN_NAME) AS
(
select name as COLUMN_NAME from sys.all_columns
where object_id = (select object_id from sys.tables where name = 'T_DataSet')
and name <> 'Data'
)
SELECT
#Columns = ISNULL(#Columns +',', '') + T.COLUMN_NAME
FROM
(
select
COLUMN_NAME = 'SUM(' +
(select SUBSTRING(
(
SELECT '+'+ COLUMN_NAME
FROM tbl_COLUMN_NAME
where LEFT(COLUMN_NAME,1) = LEFT(inner_C1.COLUMN_NAME,1)
FOR XML PATH('')
), 2 , 9999))
+ ') AS ' + LEFT(COLUMN_NAME,1)
from tbl_COLUMN_NAME as inner_C1
Group by LEFT(COLUMN_NAME,1)
)T
set #QUERY = 'select LEFT([Data],1) as Data ' + #Columns + '
From T_DataSet
Group by LEFT([Data],1)';
PRINT #QUERY
EXEC(#QUERY)
I have 14 tables of different kinds of employees. I have a C# application which is wired to my SQL Server database and when I type a last name in a textbox it brings back a record with that last name and displays it in a listbox.
However I have managed to do this with only one table. So if I type “Jones” it will bring back and display Jones from one table.
I would like to bring back all the Jones’ from all 14 tables. In other words, when I type a last name, I need the application to show me all records of that last name from all 14 tables.
What would be a reasonable approach to this? It would be a lot easier if I had one table with all employees but I need the seperation. Basically when I click the search button I need the application to go fetch from any of the 14 tables with the given name.
What would be a suitable approach to this?
Define the following stored procedure in your database:
CREATE PROCEDURE GetAll_SP
(
#FirstName VARCHAR(50)
)
AS
BEGIN
(SELECT 1, first_name, last_name FROM UsersTable1 WHERE first_name = #FirstName)
UNION
(SELECT 2, first_name, last_name FROM UsersTable2 WHERE first_name = #FirstName)
UNION
(SELECT 3, first_name, last_name FROM UsersTable3 WHERE first_name = #FirstName)
-- ....
END
GO
or the following one instead of you don't need to have any kind of control over your users location:
CREATE PROCEDURE GetAll_SP
(
#FirstName VARCHAR(50)
)
AS
BEGIN
(SELECT first_name, last_name FROM UsersTable1 WHERE first_name = #FirstName)
UNION ALL
(SELECT first_name, last_name FROM UsersTable2 WHERE first_name = #FirstName)
UNION ALL
(SELECT first_name, last_name FROM UsersTable3 WHERE first_name = #FirstName)
-- ....
END
GO
Then, in your code:
String firstName = "Jones";
using (SqlCommand cmd = new SqlCommand("GetAll_SP", m_Connection))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#FirstName", SqlDbType.VarChar).Value = firstName;
m_Connection.Open();
cmd.ExecuteNonQuery();
}
You should repeat your Code 14 Times to collect values from all this tables or you submit a SQL Query for all 14 Tables at once with unions or 14 queries in one Statement - depending on your Data Access Technology...
Okay, to get results from all 14 tables you can use UNION ALL Operator, so your SQL would look something like:
(SELECT first_name, last_name FROM table1 WHERE first_name='John')
UNION ALL
(SELECT first_name, last_name FROM table2 WHERE first_name='John')
...
(SELECT first_name, last_name FROM table14 WHERE first_name='John')
You need to have equal amount of fields in every select. However better approach would be if you save all names (and any other shared data) in one table, and connect with Key with all those 14 tables that have different dataset. That way you can prevent such a long queries like this one above (and probably slow queries) and query would look more like this:
SELECT first_name, last_name, user_type, user_id WHERE first_name='John'
And then you can retreive fields from corresponding table as field user_type gives you info in which out of 14 tables to search for other data and user_id gives you data of that user, so second query would look something like this:
SELECT job_position, worksheet, other_data FROM tableN WHERE user_id=...
Simply set #SearchStr and every column in every table will be searched.
drop table #results
go
declare #SearchStr nvarchar(100)
set #SearchStr = 'Donna%' -- use wildcards
CREATE TABLE #Results (ColumnName nvarchar(370), ColumnValue nvarchar(3630))
SET NOCOUNT ON
DECLARE #TableName nvarchar(256), #ColumnName nvarchar(128), #SearchStr2 nvarchar(110)
SET #TableName = ''
SET #SearchStr2 = QUOTENAME('%' + #SearchStr + '%','''')
WHILE #TableName IS NOT NULL
BEGIN
SET #ColumnName = ''
SET #TableName =
(
SELECT MIN(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME))
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) > #TableName
AND OBJECTPROPERTY(
OBJECT_ID(
QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)
), 'IsMSShipped'
) = 0
)
WHILE (#TableName IS NOT NULL) AND (#ColumnName IS NOT NULL)
BEGIN
SET #ColumnName =
(
SELECT MIN(QUOTENAME(COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND DATA_TYPE IN ('char', 'varchar', 'nchar', 'nvarchar', 'text')
AND QUOTENAME(COLUMN_NAME) > #ColumnName
)
print cast(#TableName as nvarchar(200)) + ' ' + #ColumnName
IF #ColumnName IS NOT NULL
BEGIN
INSERT INTO #Results
EXEC
(
--'SELECT ''' + #TableName + '.' + #ColumnName + ''', LEFT(' + #ColumnName + ', 3630)
--FROM ' + #TableName + ' (NOLOCK) ' +
--' WHERE ' + #ColumnName + ' LIKE ' + #SearchStr2
'SELECT ''' + #TableName + '.' + #ColumnName + ''', ' + #ColumnName + '
FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE ' + #ColumnName + ' LIKE ' + #SearchStr2
)
END
END
END
SELECT ColumnName, ColumnValue FROM #Results
I want to retrieve the results from SQL Server 2012.But for checking condition I rewrote the same subquery.Is there any idea to use the subquery just once and retrieve the result?
My query:
sql = "SELECT customer_id,ISNULL(first_name, '') + ' ' + ISNULL(middle_name, ' ') + ' ' + ISNULL(last_name, ' ') AS 'Customer_name', (ISNULL(city, '') + ',' + ISNULL(district, ' ') + ',' + ISNULL(zone, ' ')) as 'Location' FROM customer_detail WHERE 1=1";
if(location != "")
{
sql += " AND (ISNULL(city, '') + ',' + ISNULL(district, ' ') + ',' + ISNULL(zone, ' ')) LIKE '%" + location + "%'"";
}
Query after AND is same as above subquery. Thanks in advance.
It seems you are not looking so much for sub-query than a user defined function (UDF) to merge 3 text column in a prettier way.
If you do not want to use UDF then you could use Common Table Expressions (CTE) to write the expression just once.
Using CTE has also the benefit of encapsulating your reused block right inside query, whereas the UDF would need to be added to your DB before executing your query. This may or may not be a desirable thing depending on reusability needs.
CTE solution would be along the lines of this:
WITH CTE (Id, [Name], [Location]) as
(
SELECT customer_id,
ISNULL(first_name, '') + ' ' + ISNULL(middle_name, ' ') + ' ' + ISNULL(last_name, ' '),
ISNULL(city, '') + ',' + ISNULL(district, ' ') + ',' +ISNULL(zone, ' ')
FROM customer_detail
)
select *
from CTE
where 1=1
AND [Location] LIKE '%' + #location + '%'
Also, generally you can expect CTE to perform generally better than UDFs as the query optimizer can modify the plan to match specific query needs.
I would go with a reusable view and stored procedure for the actual search.
create view vCustomerDetail
as
select [customer_id]
, isnull([first_name] + ' ', '') + isnull([middle_name] + ' ', '') + isnull([last_name], '') as [Customer_name]
, isnull([city] + ', ', '') + isnull([district] + ', ', '') + isnull([zone], '') as [Location]
from [customer_detail]
go
create proc pCustomerDetailByLocation
(
#location nvarchar(200) = ''
)
as
begin
set nocount on;
select *
from [vCustomerDetail]
where [Location] = ''
union all
select *
from [vCustomerDetail]
where [Location] like '%' + #location + '%';
end
go
You would call the stored procedure from your code with and pass along the location in the parameter. That way you can prevent SQL injection and also make use of better execution plan caching than with an ad-hoc query.
You can either use a union like I've done in my example, or combine it with an OR statement, like so:
where [Location] = '' or [Location] like '%' + #location + '%';
I found the union to be beneficial in some cases over because you're helping the optimizer split up his lookup logic.
I have a stored procedure which results in lots of data. and also want to convert this to EF
unable to figure out how to join to the relavent tables when an attribute is present for the system. and also the column selection is very dynamic in nature,
I could take this sql and execute this directly and get things sorted that way but would miss but the grid in the front end wont be able to handle 600mb of data thrown from the database.
so need paging thought can do this better with EF.
for reference purpose I have the following sql below.
Declare #SQL varchar(max);
Declare #SelectColumns VARCHAR(MAX)
SELECT DISTINCT #SelectColumns= STUFF((SELECT ',''' + [PrimaryDataSource] + ''' Golden'
+ ISNULL(CASE WHEN System1 IS NOT NULL THEN ', System1.' + QUOTENAME([System1]) + ' System1' END, '')
+ ISNULL(CASE WHEN System2 IS NOT NULL THEN ', System2.' + QUOTENAME([System2]) + ' System2' END, '')
+ ISNULL(CASE WHEN [System3] IS NOT NULL THEN ', System3.' + QUOTENAME([System3])+ ' System3' END, '')
+ ISNULL(CASE WHEN System4 IS NOT NULL THEN ', System4.' + QUOTENAME(System4)+ ' System4' END, '')
+ ISNULL(CASE WHEN System5 IS NOT NULL THEN ', System5.' + QUOTENAME(System5)+ ' System5' END, '')
+ ISNULL(CASE WHEN System6 IS NOT NULL THEN ', System6.' + QUOTENAME(System6)+ ' System6' END, '')
FROM [dbo].[TBL_Mapping]
where Attribute =#attributeName
FOR XML PATH(''), TYPE
).value('.', 'VARCHAR(MAX)')
,1,1,'')
SET #SQL = '
SELECT distinct
m.ID MappingID,
m.KeyValueUniqueKey,
m.ValueKeyUniqueKey,
' + #SelectColumns + '
FROM [dbo].[TBL_Mapping] M '
IF CHARINDEX('System1.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL +
'
LEFT OUTER JOIN dbo.VW_System1_ALL System1 ON
System1.System1ID=M.System1ID '
END
IF CHARINDEX('System2.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL +
'
LEFT OUTER JOIN dbo.TBL_System2 System2 ON
M.System2ID= System2.System2ID '
END
IF CHARINDEX('System4.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN DBO.tbl_System4 System4 ON
System4.Key1 = M.KeyValueUniqueKey AND
System4.Value1 = ValueKeyUniqueKey '
END
IF CHARINDEX('System5.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN DBO.tbl_System5 System5 ON
System5.System5Id = M.System5Id'
END
IF CHARINDEX('System6.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN dbo.tbl_system6 System6 ON
System6.System6Id = M.System6Id'
END
IF CHARINDEX('System3.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN [dbo].[TBL_System3] System3 ON
System3.System3Id = M.System3Id'
END
SET #SQL = #SQL + '
WHERE m.version=0 and isActive=1
ORDER by m.ID'
print #SQL
exec (#SQL)
I have looked at the Leftjoin2 extn method but that is not helping much.
What is the best possible action to get this on to EF.
or EF itself is a wrong choise for this sort of problems?
You can do dynamic query generating and then in the end do Skip().Take().
Your model for custom object may look like this:
class MappingData
{
//not sure what the data types are.
int MappingId;
int KeyValueUniqueKey;
int ValueKeyUniqueKey;
string System1;
string System2;
...
string System6;
}
Then in the get method map data,
IQueryable<MappingData> sql = db.TBL_Mapping
.Select(m => new MappingData {
MappingId = ID,
KeyValueUniqueKey = KeyValueUniqueKey,
ValueKeyUniqueKey = ValueKeyUniqueKey,
//leave other columns out
//they will be filled in
//dynamically
})
.Distinct();//get distinct
//--------------------
//REPEAT START
bool HasSystem1 = db.TBL_Mapping.Any(m => m.System1 != null);
//left outer join with System1 if it has it in the TBL_Mapping
if (HasSystem1)
{
sql =
from m in sql
join s1 in db.VW_System1_ALL
on m.System1ID equals s1.System1ID into stemp
from st in stemp.DefaultIfEmpty()
select new { MappingId = st.Id,
KeyValueUniqueKey = st.KeyValueUniqueKey,
ValueKeyUniqueKey = st.ValueKeyUniqueKey,
System1 = st.System1 }; //SystemX column.
}
//REPEAT END
//--------------------
// repeat the above for System2 thru System6
//And in the end do paging.
var result = sql
.Skip(currentPageNumber * numberOfObjectsInPage)
.Take(numberOfObjectsInPage);
This is a bad fit for EF. If all you are only trying to add paging -- add your own paging functionality to the stored proc. You can do this by using ROW_NUMBER OVER what every you are sorting by, then use an an outer query to return the page of data you want, for example...
CREATE PROCEDURE [dbo].[PagedSomething]
#pageSize int,
#pageNum int -- assume pages are 0-based
AS
BEGIN
-- outer query does the paging in its where clause,
-- returning the selected "pages" from the raw results of the inner query
SELECT RawResults.SomethingId
FROM
-- inner query where you make your basic data
(SELECT
s.SomethingId
, ROW_NUMBER() OVER(ORDER BY s.SomethingId) RowID
FROM Somethings s) RawResults
WHERE RowID >= #pageNum * #pageSize + 1
AND RowID < (#pageNum + 1) * #pageSize + 1
END