I have a stored procedure which results in lots of data. and also want to convert this to EF
unable to figure out how to join to the relavent tables when an attribute is present for the system. and also the column selection is very dynamic in nature,
I could take this sql and execute this directly and get things sorted that way but would miss but the grid in the front end wont be able to handle 600mb of data thrown from the database.
so need paging thought can do this better with EF.
for reference purpose I have the following sql below.
Declare #SQL varchar(max);
Declare #SelectColumns VARCHAR(MAX)
SELECT DISTINCT #SelectColumns= STUFF((SELECT ',''' + [PrimaryDataSource] + ''' Golden'
+ ISNULL(CASE WHEN System1 IS NOT NULL THEN ', System1.' + QUOTENAME([System1]) + ' System1' END, '')
+ ISNULL(CASE WHEN System2 IS NOT NULL THEN ', System2.' + QUOTENAME([System2]) + ' System2' END, '')
+ ISNULL(CASE WHEN [System3] IS NOT NULL THEN ', System3.' + QUOTENAME([System3])+ ' System3' END, '')
+ ISNULL(CASE WHEN System4 IS NOT NULL THEN ', System4.' + QUOTENAME(System4)+ ' System4' END, '')
+ ISNULL(CASE WHEN System5 IS NOT NULL THEN ', System5.' + QUOTENAME(System5)+ ' System5' END, '')
+ ISNULL(CASE WHEN System6 IS NOT NULL THEN ', System6.' + QUOTENAME(System6)+ ' System6' END, '')
FROM [dbo].[TBL_Mapping]
where Attribute =#attributeName
FOR XML PATH(''), TYPE
).value('.', 'VARCHAR(MAX)')
,1,1,'')
SET #SQL = '
SELECT distinct
m.ID MappingID,
m.KeyValueUniqueKey,
m.ValueKeyUniqueKey,
' + #SelectColumns + '
FROM [dbo].[TBL_Mapping] M '
IF CHARINDEX('System1.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL +
'
LEFT OUTER JOIN dbo.VW_System1_ALL System1 ON
System1.System1ID=M.System1ID '
END
IF CHARINDEX('System2.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL +
'
LEFT OUTER JOIN dbo.TBL_System2 System2 ON
M.System2ID= System2.System2ID '
END
IF CHARINDEX('System4.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN DBO.tbl_System4 System4 ON
System4.Key1 = M.KeyValueUniqueKey AND
System4.Value1 = ValueKeyUniqueKey '
END
IF CHARINDEX('System5.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN DBO.tbl_System5 System5 ON
System5.System5Id = M.System5Id'
END
IF CHARINDEX('System6.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN dbo.tbl_system6 System6 ON
System6.System6Id = M.System6Id'
END
IF CHARINDEX('System3.',#SelectColumns) > 0
BEGIN
SET #SQL = #SQL + '
LEFT OUTER JOIN [dbo].[TBL_System3] System3 ON
System3.System3Id = M.System3Id'
END
SET #SQL = #SQL + '
WHERE m.version=0 and isActive=1
ORDER by m.ID'
print #SQL
exec (#SQL)
I have looked at the Leftjoin2 extn method but that is not helping much.
What is the best possible action to get this on to EF.
or EF itself is a wrong choise for this sort of problems?
You can do dynamic query generating and then in the end do Skip().Take().
Your model for custom object may look like this:
class MappingData
{
//not sure what the data types are.
int MappingId;
int KeyValueUniqueKey;
int ValueKeyUniqueKey;
string System1;
string System2;
...
string System6;
}
Then in the get method map data,
IQueryable<MappingData> sql = db.TBL_Mapping
.Select(m => new MappingData {
MappingId = ID,
KeyValueUniqueKey = KeyValueUniqueKey,
ValueKeyUniqueKey = ValueKeyUniqueKey,
//leave other columns out
//they will be filled in
//dynamically
})
.Distinct();//get distinct
//--------------------
//REPEAT START
bool HasSystem1 = db.TBL_Mapping.Any(m => m.System1 != null);
//left outer join with System1 if it has it in the TBL_Mapping
if (HasSystem1)
{
sql =
from m in sql
join s1 in db.VW_System1_ALL
on m.System1ID equals s1.System1ID into stemp
from st in stemp.DefaultIfEmpty()
select new { MappingId = st.Id,
KeyValueUniqueKey = st.KeyValueUniqueKey,
ValueKeyUniqueKey = st.ValueKeyUniqueKey,
System1 = st.System1 }; //SystemX column.
}
//REPEAT END
//--------------------
// repeat the above for System2 thru System6
//And in the end do paging.
var result = sql
.Skip(currentPageNumber * numberOfObjectsInPage)
.Take(numberOfObjectsInPage);
This is a bad fit for EF. If all you are only trying to add paging -- add your own paging functionality to the stored proc. You can do this by using ROW_NUMBER OVER what every you are sorting by, then use an an outer query to return the page of data you want, for example...
CREATE PROCEDURE [dbo].[PagedSomething]
#pageSize int,
#pageNum int -- assume pages are 0-based
AS
BEGIN
-- outer query does the paging in its where clause,
-- returning the selected "pages" from the raw results of the inner query
SELECT RawResults.SomethingId
FROM
-- inner query where you make your basic data
(SELECT
s.SomethingId
, ROW_NUMBER() OVER(ORDER BY s.SomethingId) RowID
FROM Somethings s) RawResults
WHERE RowID >= #pageNum * #pageSize + 1
AND RowID < (#pageNum + 1) * #pageSize + 1
END
Related
ALTER PROCEDURE [dbo].[ListaUnidadAlumnos]
#MateriaId INT,
#UnidadId INT,
#Columnas AS NVARCHAR(MAX) = Null,
#Query AS NVARCHAR(MAX) = Null
AS
BEGIN
SET NOCOUNT ON;
SELECT
#Columnas = STUFF((SELECT ',' + QUOTENAME(convert(char(10), L.Fecha, 120))
FROM (SELECT DISTINCT(A.Fecha)
FROM Asistencias A
INNER JOIN AspNetUsers U ON A.AlumnoId = U.Id
INNER JOIN Unidades UN ON UN.UnidadId = A.UnidadId
INNER JOIN Materias M ON UN.MateriaId = M.MateriaId
WHERE UN.MateriaId = #MateriaId
AND UN.UnidadId = #UnidadId) L
GROUP BY L.Fecha
ORDER BY L.Fecha asc
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
SET #Query = 'SELECT AlumnoId, ' + #Columnas + ' from
(
SELECT A.AlumnoId, A.Fecha, A.Valor
FROM Asistencias A
INNER JOIN AspNetUsers U ON A.AlumnoId = U.Id
) x
pivot
(
max(Valor)
for Fecha in (' + #Columnas + ')
) p '
EXECUTE(#query)
This stored procedure returns:
The dates are dynamic, depending on the assistance of the class
I am not using entity framework, I execute the procedures in the following way
public async Task ObtenerListaMateria(int materiaId)
{
var pMateriaId = new SqlParameter("#MateriaId", materiaId);
ListaAlumnosIdMateria = await _context.ListaAlumnosIdMateria
.FromSql("EXEC ObtenerListaAlumnosIdMateria #MateriaId", pMateriaId)
.ToListAsync();
ViewData["NumeroAlumnos"] = ListaAlumnosIdMateria.Count();
ViewData["Contador"] = 0;
}
If I execute the procedures in this way, first I have to create the model with the data that they see in the procedure
My question is, how do you call the procedure if the query will always give me a different number of dates, can I create a generic model?
I have this Linq query:
IQueryable<SPR> query = db.SPRs;
if (!string.IsNullOrEmpty(search.accountNumber))
{
query = query.Where(b => b.CustomerAccountNumber.Contains(search.accountNumber));
}
if (!string.IsNullOrEmpty(search.accountName))
{
query = query.Where(b => b.CustomerNumber.Contains(search.accountName));
}
if (!string.IsNullOrEmpty(search.submittedBy))
{
query = query.Where(b => b.SubmittedBy.Contains(search.submittedBy));
}
if (!string.IsNullOrEmpty(search.smName))
{
query = query.Where(b => b.SMUserName == search.smName);
}
var result = query.ToList();
I am just appending the where clause if conditions are true. The issue is that it is not just adding a And in the generated SQL where clause like I want it to.
Here is the generated SQL if I have the SubmittedBy and SMUserName filled with data.
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[CustomerNumber] AS [CustomerNumber],
[Extent1].[CustomerAccountNumber] AS [CustomerAccountNumber],
[Extent1].[SMUserName] AS [SMUserName],
[Extent1].[SubmittedBy] AS [SubmittedBy],
[Extent1].[Notes] AS [Notes]
FROM
[dbo].[SPRs] AS [Extent1]
WHERE
([Extent1].[SubmittedBy] LIKE #p__linq__0 ESCAPE N'~')
AND (([Extent1].[SMUserName] = #p__linq__1) OR (([Extent1].[SMUserName] IS NULL)
AND (#p__linq__1 IS NULL)))
Not sure how this last line OR (([Extent1].[SMUserName] IS NULL) AND (#p__linq__1 IS NULL))) is getting added which is messing the query up.
Can someone please tell me how I can have just AND in the eventual query when the if conditions are satisfied?
Since you are working with sql server a more performance efficient and sleek way would be to handle the optional parameters inside a stored procedure and make use of Dynamic sql with sp_executesql to benefit from Parameterised Execution Plans.
CREATE PROCEDURE getSPR
#SubmittedBy Varchar(100) = NULL --<--- Use appropriate datatypes
,#CustomerAccountNumber Varchar(100) = NULL
,#CustomerNumber Varchar(100) = NULL
,#SMUserName Varchar(100) = NULL
AS
BEGIN
SET NOCOUNT ON;
Declare #Sql Nvarchar(max);
SET #Sql = N'SELECT [Id]
,[CustomerNumber]
,[CustomerAccountNumber]
,[SMUserName]
,[SubmittedBy]
,[Notes]
FROM [dbo].[SPRs]
WHERE 1 = 1 '
+ CASE WHEN #SubmittedBy IS NOT NULL THEN
N' AND [SubmittedBy] LIKE ''%'' + #SubmittedBy + ''%''' ELSE N' ' END
+ CASE WHEN #CustomerAccountNumber IS NOT NULL THEN
N' AND [CustomerAccountNumber] LIKE ''%'' + #CustomerAccountNumber + ''%''' ELSE N' ' END
+ CASE WHEN #CustomerNumber IS NOT NULL THEN
N' AND [CustomerNumber] LIKE ''%'' + #CustomerNumber + ''%''' ELSE N' ' END
+ CASE WHEN #SMUserName IS NOT NULL THEN
N' AND [SMUserName] = #SMUserName ' ELSE N' ' END
Exec sp_executesql #sql
,N' #SubmittedBy Varchar(100),#CustomerAccountNumber Varchar(100)
,#CustomerNumber Varchar(100), #SMUserName Varchar(100)'
,#SubmittedBy
,#CustomerAccountNumber
,#CustomerNumber
,#SMUserName
END
Some notes:
ExecuteNonQuery returns -1
ExecuteNonQuery will drop the table (#droptable), but it will not create the new table (#code)
the length of the #code query is 10265 characters
The stored procedure runs perfectly fine in SSMS and returns 22 rows in the table
Are there any ideas as to why C#'s ExecuteNonQuery function doesn't seem to be executing the 'exec(#code)' portion of the stored procedure?
ALTER procedure [dbo].[sp_create_EditControlResultsPivot]
as
begin
declare #t nvarchar (250);
set #t = 'editControlResults'
declare #newtable nvarchar(250);
set #newtable = 'dbo.' + #t + 'Pivot'
declare #nonPivotColumn1 nvarchar(250);
set #nonPivotColumn1 = 'num'
declare #nonPivotColumn2 nvarchar(25);
set #nonPivotColumn2 = 'File_Name'
declare #droptable nvarchar(max);
set #droptable =
'if EXISTS (select * from sys.objects where object_id = object_id(N''' + #newtable + '''))
begin drop table ' + #newtable + ' end
'
declare #i int
set #i = 1;
declare #itemList nvarchar(max);
declare #code nvarchar(max);
while #i <= (
select COUNT(*)
from sys.columns c
left join sys.tables t on c.object_id = t.object_id
where 1=1
and c.name not like #nonPivotColumn1
and c.name not like #nonPivotColumn2
and t.name = #t
)
begin
set #itemList = #itemList + ', ' +
(
select col from
(
select c.name as col, ROW_NUMBER () over (order by c.name) as num from
sys.columns c left join sys.tables t on c.object_id = t.object_id
where 1=1
and c.name not like #nonPivotColumn1
and c.name not like #nonPivotColumn2
and t.name = #t
) sub where num = #i
)
set #i = #i + 1
end
set #itemList = (select substring(#itemList, 2, LEN(#itemList)))
set #code = '
SELECT ' + #nonpivotcolumn2 + ', Item
into ' + #newtable + '
FROM
(SELECT ' + #nonpivotcolumn2 + ', ' + #itemList + '
FROM ' + #t + ') sub
UNPIVOT
(Value FOR Item IN (' + #itemList + ')
) AS sub
where Value = ''true''
'
exec(#droptable)
exec(#code);
--print(len(#code))
END
--exec sp_create_EditControlResultsPivot
The ExecuteNonQuery Method returns the number of rows affected use the ExecuteReader method instead.
SqlCommand.ExecuteReader Method
The only way to return data from ExecuteNonQuery would be via an Output parameter.
I suspect your comment #3. the length of the #code query is 10265 characters...could be an issue...I think the call from C# is chopping it to only 4000 or 8000 chars...
Since you are not expecting a resultset, ExecuteNonQuery is good.
Things to try:
Try inserting the content of the #code variable (inside the procedure) in a table and see if you are getting the correct sql...both when executed from SSMS and from C# call
If you get a valid sql query in step 1 (which I doubt)...try executing that query in SSMS to see if it really works...
We're looking to do an update in several SQL Server databases to change all NULL values in a certain table to be empty strings instead of NULL. We're potentially going to be doing this across hundreds of databases. The table name will always be the same, but the column names are variable based on how the front-end application is configured (don't judge... I didn't create this system).
Is there a way to do an update on all of these columns without knowing the column names ahead of time?
You can pass the name of the column in dynamic sql:
declare #sql nvarchar (1000);
set #sql = N'update table set ' + #column_name + '= ''''';
exec sp_executesql #sql;
You can look in the sys.columns table and join on the table name or object_id.
DECLARE #OBJ_ID INT
SELECT #OBJ_ID = OBJECT_ID
FROM SYS.tables
WHERE name = 'YOURTABLE'
SELECT * FROM SYS.columns
WHERE OBJECT_ID = #OBJ_ID
You could use the name field from the sys.columns query as a basis to perform the update on.
Assuming you want all columns of varchar/char types only (or change the type filter to whatever you need):
DECLARE #tableName varchar(10)
SET #tableName = 'yourtablenamehere'
DECLARE #sql VARCHAR(MAX)
SET #sql = ''
SELECT #sql = #sql + 'UPDATE ' + #tableName + ' SET ' + c.name + ' = '''' WHERE ' + c.name + ' IS NULL ;'
FROM sys.columns c
INNER JOIN sys.tables t ON c.object_id = t.object_id
INNER JOIN sys.types y ON c.system_type_id = y.system_type_id
WHERE t.name = #tableName AND y.name IN ('varchar', 'nvarchar', 'char', 'nchar')
EXEC (#sql)
This can be achieved with cursors. You first select the column names like #Darren mentioned, then you Set a Cursor with those values and loop:
Open oColumnsCursor
Fetch Next From oColumnscursor
Into #ColumnName
While ##FETCH_STATUS=0
Begin
Set #oQuery = 'Update [DB]..[Table] Set [' + #ColumnName + '] = ''NewValue'' Where [' + #ColumnName + '] = ''OldValue'''
Execute(#oQuery)
Fetch Next From oColumnscursor Into #ColumnName
Set #oCount = #oCount + 1
End
Close oColumnsCursor;
Deallocate oColumnsCursor;
This will work when you know the Table Name:
DECLARE #tableName varchar(10)
SET #tableName = 'Customers'
DECLARE #sql VARCHAR(MAX)
SET #sql = ''
SELECT #sql = #sql + 'UPDATE ' + #tableName + ' SET ' + c.name + ' = ISNULL('+ c.name +','''');'
FROM sys.columns c
INNER JOIN sys.tables t ON c.object_id = t.object_id
INNER JOIN sys.types y ON c.system_type_id = y.system_type_id
WHERE y.name IN ('varchar', 'nvarchar', 'char', 'nchar')
AND t.name = #tableName;
EXEC(#sql);
And this will iterate all Tables and all Columns in a Db:
DECLARE #sql VARCHAR(MAX)
SET #sql = ''
SELECT #sql = #sql + 'UPDATE ' + t.name + ' SET ' + c.name + ' = ISNULL('+ c.name +','''');'
FROM sys.columns c
INNER JOIN sys.tables t ON c.object_id = t.object_id
INNER JOIN sys.types y ON c.system_type_id = y.system_type_id
WHERE y.name IN ('varchar', 'nvarchar', 'char', 'nchar');
EXEC(#sql);
Below is the procedure.
ALTER PROCEDURE [dbo].[util_db_updateRow]
#colval_name NVARCHAR (30), -- column and values e.g. tax='5.50'
#idf_name NVARCHAR (300), -- column name
#idn_name NVARCHAR (300), -- column value
#tbl_name NVARCHAR (100) -- table name
AS
BEGIN
SET NOCOUNT ON;
DECLARE #sql NVARCHAR(MAX)
-- construct SQL
SET #sql = 'UPDATE ' + #tbl_name + ' SET ' + #colval_name +
' WHERE ' + #idf_name + '=' + #idn_name;
-- execute the SQL
EXEC sp_executesql #sql
SET NOCOUNT OFF
RETURN
END
Below is the stored procedure where you can pass Schema Name, Table Name and list of column names separted by comma.It works only in Sql Server 2016 or higher.
CREATE OR ALTER PROCEDURE UpdateData
(#SchemaName NVARCHAR(Max),#TableName NVARCHAR(MAX),#ColumnNames NVARCHAR(MAX))
AS
BEGIN
DECLARE #DynamicSql NVARCHAR(MAX);
SET #DynamicSql = 'UPDATE ' +'[' +#SchemaName+'].' + '[' +#TableName+']' +' SET ' + STUFF((SELECT ', [' + C.name + '] = ' + '''NEW_VALUE'''
FROM sys.columns C
INNER JOIN sys.tables T ON T.object_id = C.object_id
INNER JOIN sys.schemas S ON T.schema_id = S.schema_id
WHERE
T.name = #TableName
AND S.Name = #SchemaName
AND [C].[name] in (SELECT VALUE FROM string_split(#ColumnNames,','))
FOR XML PATH('')), 1,1, '')
print #DynamicSql;
EXEC (#DynamicSql);
END
I have a stored procedure which gives different result in only a specific case.
When I call it from SQL Server Management Studio 2008 R2, it gives me 0 as output.
When I call it from C# class file. It gives me 1 as output.
I am using edmx file, and it is updated for sure.
The call is something like below from SSMS [SQL Server Management Studio]
exec proc_GetPrimaryKeyUsageCount 62, 'tblFormula'
This gives output as 0
The same stored procedure is called from C# file is like below
_db.GetPrimaryKeyUsageCount(62, "tblFormula");
This gives output as 1
The stored procedure is
CREATE PROCEDURE proc_GetPrimaryKeyUsageCount (
#PrimaryKeyColumnId INT
,#PrimaryKeyTable NVARCHAR(max)
--,#Response INT OUTPUT
)
AS
BEGIN
DECLARE #counter INT
DECLARE #sqlCommand NVARCHAR(max)
DECLARE #ForeignKey TABLE (
child_table VARCHAR(max)
,child_fk_column VARCHAR(max)
)
DECLARE #child_table VARCHAR(max)
DECLARE #child_fk_column VARCHAR(max)
SET #counter = 0
INSERT INTO #ForeignKey
SELECT child_table = c.TABLE_NAME
,child_fk_column = c.COLUMN_NAME
FROM INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE p
INNER JOIN INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS pc ON pc.UNIQUE_CONSTRAINT_SCHEMA = p.CONSTRAINT_SCHEMA
AND pc.UNIQUE_CONSTRAINT_NAME = p.CONSTRAINT_NAME
INNER JOIN INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE c ON c.CONSTRAINT_SCHEMA = pc.CONSTRAINT_SCHEMA
AND c.CONSTRAINT_NAME = pc.CONSTRAINT_NAME
WHERE EXISTS (
SELECT 1
FROM INFORMATION_SCHEMA.COLUMNS
WHERE COLUMN_NAME = 'IsDeleted'
AND TABLE_SCHEMA = p.TABLE_SCHEMA
AND TABLE_NAME = p.TABLE_NAME
AND p.TABLE_NAME = #PrimaryKeyTable
)
DECLARE db_cursor CURSOR
FOR
SELECT child_table
,child_fk_column
FROM #ForeignKey
OPEN db_cursor
FETCH NEXT
FROM db_cursor
INTO #child_table
,#child_fk_column
WHILE ##FETCH_STATUS = 0
BEGIN
PRINT 'select count(*) from ' + CAST(#child_table AS VARCHAR) + ' where ' + CAST(#child_fk_column AS VARCHAR) + ' = ' + CAST(#PrimaryKeyColumnId AS VARCHAR)
SET #sqlCommand = 'select #cnt=count(*) from ' + CAST(#child_table AS VARCHAR) + ' where ' + CAST(#child_fk_column AS VARCHAR) + ' = ' + CAST(#PrimaryKeyColumnId AS VARCHAR)
EXEC sp_executesql #sqlCommand
,N'#cnt int OUTPUT'
,#cnt = #counter OUTPUT
IF #counter > 0
BREAK
FETCH NEXT
FROM db_cursor
INTO #child_table
,#child_fk_column
END
SELECT #counter AS [PrimaryKeyUsageCount]
END
1st argument is Id of the primary key and 2nd argument is the name of the table having that primary key.
The Procedure returns the count of the usage of primary key in other tables in same database. If it finds even 1 occurrence, it will return that count otherwise 0.
If anything extra is needed please do let me know.
There are couple of mistakes, which could cause the problem.
The INSERT should be like that:
INSERT INTO #ForeignKey
SELECT c.TABLE_NAME,c.COLUMN_NAME
FROM INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE p
INNER JOIN INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS pc ON pc.UNIQUE_CONSTRAINT_SCHEMA = p.CONSTRAINT_SCHEMA
AND pc.UNIQUE_CONSTRAINT_NAME = p.CONSTRAINT_NAME
INNER JOIN INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE c ON c.CONSTRAINT_SCHEMA = pc.CONSTRAINT_SCHEMA
AND c.CONSTRAINT_NAME = pc.CONSTRAINT_NAME
WHERE EXISTS (
SELECT *
FROM INFORMATION_SCHEMA.COLUMNS AS isc
WHERE isc.COLUMN_NAME = 'IsDeleted'
AND isc.TABLE_SCHEMA = p.TABLE_SCHEMA
AND isc.TABLE_NAME = p.TABLE_NAME
AND p.TABLE_NAME = #PrimaryKeyTable
)
After cursor loop shoud be:
CLOSE db_cursor
DEALLOCATE db_cursor