I have a gridview that can be filtered by 6 drop down boxes, so when writing the sql the easiest thing would be to use an 'or' statement if dropdown has selection or null etc.
However I have read on here and other sites that using sql or statements are a bad idea, can anyone offer any other suggestions i could use rather than me writing variations on whether each ddl selection is null? Below is an example of the first query, with every ddl returning a value
#ruleID int = null,
#engagementStatusID int = null,
#areaOfWorkID int = null,
#registered bit = null,
#staffGroupID int = null,
#assignmentTypeID int = NULL
AS
SET NOCOUNT ON
IF (#ruleID IS NOT NULL and #engagementStatusID IS NOT NULL and #areaOfWorkID IS NOT NULL and
#registered IS NOT NULL and #staffGroupID IS NOT NULL and #assignmentTypeID IS NOT NULL)
BEGIN
SELECT r.dbRuleId AS RuleID,r.dbEngagementStatusId AS EngagementStatusID,
r.dbIsAllStaffGroups AS AllStaffGroups,r.dbIsAllAssignments AS AllAssignments,
r.dbIsAllRegistered AS AllRegistered,r.dbIsAllUnregistered AS AllUnregistered,
r.dbSoftDelete AS Softdelete, es.dbName AS EngagementName,
sgc.dbName AS StaffGroupName, aow.dbName AS AreaOfWorkName,
at.dbName AS AssignmentName, at.dbIsRegistered AS Registered,sgc.dbStaffGroupCodeId AS StaffGroupCodeID,
at.dbAssignmentTypeId AS AssignmentID, aow.dbAreaOfWorkId AS AreaOfWorkID
FROM dbo.tbRule r INNER JOIN
dbo.EngagementStatus es ON r.dbEngagementStatusId = es.dbEngagementStatusId INNER JOIN
dbo.RuleStaffGroup rsg ON r.dbRuleId = rsg.dbRuleId INNER JOIN
dbo.StaffGroupCode sgc ON rsg.dbStaffGroupId = sgc.dbStaffGroupCodeId INNER JOIN
dbo.RuleAssignmentCode rac ON r.dbRuleId = rac.dbRuleId INNER JOIN
dbo.AssignmentCode ac ON
rac.dbAssignmentCodeId = ac.dbAssignmentCodeId INNER JOIN
dbo.AssignmentType at ON ac.dbAssignmentId = at.dbAssignmentTypeId INNER JOIN
dbo.AreaOfWork aow ON ac.dbAreaOfWorkId = aow.dbAreaOfWorkId
WHERE ((r.dbRuleId = #ruleID) and (r.dbEngagementStatusId = #engagementStatusID) and (aow.dbAreaOfWorkId = #areaOfWorkID) and
(at.dbIsRegistered = #registered) and (sgc.dbStaffGroupCodeId = #staffGroupID) and (at.dbAssignmentTypeId = #assignmentTypeID))
Any advice on this would be great
Update
I feel i should clarify something about my code, when i say null, this is the value i have assigned to the "all" selection of a drop down list, so for example imn most cases i do something like this to get the value that needs to be passed to the DB
int? Type = (this.ddlType.SelectedValue.ToString() == "All") ? (int?)null : Convert.ToInt32(this.ddlType.SelectedValue.ToString());
so if the user has selected all the Db recieves 'null' which i can then use on the 'if #blah IS NOT NULL' etc. I realise this is probably not the best way to do this
It seems you are executing this stored procedure and then validating the user input at the database level. You should not call the database stored procedure if the drop down list values are null, you can handle this at the client side (or server side).
Client side (JavaScript) would be better for the users experience, you can then invoke the stored procedure if the user has selected all appropriate drop down list values.
The problem comes when you do things like:
WHERE (r.dbRuleId = #ruleID or #ruleID is null)
and (r.dbEngagementStatusId = #engagementStatusID
or #engagementStatusID is null)
-- ... lots more
which quickly degrades to really bad query plans. The trick, then, is to have TSQL that matches your exact set of query parameters.
The hard to maintain way to fix this is to write DML for every single possibility, and branch into the correct one - but this is really ugly, and confuses a lot of tools.
The easiest way to do this is to build the TSQL appropriately at the caller - but if your system demands that you use a stored procedure (the benefits for which, these days, are dubious at best - btw), then the simplest choice is dynamic SQL. Obviously, you need to be careful here - you still don't want to concatenate inputs (for both injection and query-plan reasons), but - you can do something like:
declare #sql nvarchar(4000) = N'...start of query...';
if(#ruleID is not null)
set #sql = #sql + N' and r.dbRuleId = #ruleID';
if(#engagementStatusID is not null)
set #sql = #sql + N' and r.dbEngagementStatusId = #engagementStatusID';
You then need to execute that with sp_executesql, declaring the parameters:
exec 'sp_executesql', #sql,
N'#ruleID int, #engagementStatusID int',
#ruleID, #engagementStatusID
I'm not sure I understood your question, but if you're looking at avoiding repetition of OR operators, consider using IN('x','y','z') - listing the possible values. This would be easier to read than [something] = 'x' OR [something] = 'y' OR [something] = 'z'.
Related
I am trying to get SQL to list not found when a machine name lookup is not in the database. I have the query working but I need to integrate it into C# where I have other queries. As it stands it is a lot more convoluted than I want it to be, I am guessing there is a much more intelligent and concise why of having the SQL achieve the same result.
DECLARE #myHostCount as INT
DECLARE #myHostName1 as Varchar(50);
DECLARE #myHostName2 as Varchar(50) = 'Machine220054';
DECLARE #myHostCount1 as INT
DECLARE #myHostName3 as Varchar(50);
DECLARE #myHostName4 as Varchar(50) = 'Machine22054';
SET #myHostCount1 = (SELECT COUNT(*) FROM db_owner.host WHERE name LIKE '%Machine22054%')
SET #myHostName3 = (SELECT Name FROM db_owner.host WHERE name LIKE '%Machine22054%')
SET #myHostCount = (SELECT COUNT(*) FROM db_owner.host WHERE name LIKE '%Machine220054%')
SET #myHostName1 = (SELECT Name FROM db_owner.host WHERE name LIKE '%Machine220054%')
SELECT
CASE
WHEN #myHostCount = 1 THEN 'Found'
WHEN #myHostCount = 0 THEN 'Not Found'
END AS 'Result',
#myHostName2 AS 'HostName'
UNION
SELECT
CASE
WHEN #myHostCount1 = 1 THEN 'Found'
WHEN #myHostCount1 = 0 THEN 'Not Found'
END AS 'Result',
#myHostName4 AS 'HostName'
Output:
Result Machine Name
--------------------------
Found Machine220054
Not Found Machine22054
You can try this:
DECLARE #myHostName2 as Varchar(50) = 'Machine220054';
DECLARE #myHostName4 as Varchar(50) = 'Machine22054';
DECLARE #hostName TABLE (
Name VARCHAR(255)
);
INSERT INTO #hostName (Name)
VALUES (#myHostName2), (#myHostName4)
SELECT HostName = HostName.name,
Result = IIF(COUNT(*) = 0, 'Not Found', 'Found')
FROM #hostName AS HostName
LEFT JOIN db_owner.host ON host.name LIKE '%' + HostName.Name + '%'
GROUP BY HostName.name;
First: Try to remove as much procedural logic in SQL as possible.
Second: Try to minimize the use of "Like" queries if at all possible. It will hinder indexing and cause bottlenecks eventually, or, at least limit it to ending with a "%" instead of starting it with one.
Third: you are assuming that there will only be zero or one hit in your code. What if (given either non-unique or like based queries) there are more than one?
I'd go for something like this:
declare #looking table (name varchar(50))
insert into #looking (name) values ('Machine220054'),('Machine22054')
select case when t.name is null then 'Not Found' else 'Found' end as [FoundOrNot], t.name as [FoundName],l.name as [SearchName]
from #looking as l
left outer join db_owner.host as t
on t.name like '%' + l.name + '%'
UPDATE: Tested against a random table, removed "group by" on account of either one or no hit expected.
Just to provide an update, I ended up taking the results from the DR reader and entering them into a datatable, since the dr reader can only be read once. Then I compared the selections from the listbox with the contents from the datatable then added the difference to another datatable then displayed the results in a datagrid, this produced the desired outcome of results found, being listed and the objects not found being listed as not found.
Perhaps I'm misunderstanding COALESCE, but in essence, what I'm trying to do is run a select query that if it returns NULL instead of an int, instead return 0 for the purposes of ExecuteScalar().
SQL Server query:
SELECT TOP 1 COALESCE(SeqNo,0)
FROM tblProjectChangeOrder
WHERE ProjectID = XXXXX
ORDER BY SeqNo DESC
If the supplied ProjectID exists in the Change Order table, it returns the expected highest SeqNo. However, if the supplied ProjectID has no existing Change Orders (thus returns NULL for SeqNo), rather than the COALESCE returning 0, I am still getting NULL.
Am I just getting the syntax wrong or is what I want to do possible with COALESCE? The other option I see is to have my ExecuteScalar() pass to a nullable int, then follow that with a ?? to coalesce in my C# codebehind.
As john has mentioned in the comments, COALESCE operates at row level. If a table contains no rows, or a statement returns no rows, then no rows will be returned. Take the simple example below:
CREATE TABLE #Sample (ID int);
SELECT COALESCE(ID, 0)
FROM #Sample;
DROP TABLE #Sample;
Notice that nothing is returned.
Instead, one method is to use a subquery. For your query, that would result in:
SELECT COALESCE(SELECT TOP 1 SeqNo
FROM tblProjectChangeOrder
WHERE ProjectID = XXXXX
ORDER BY SeqNo DESC),0) AS SeqNo;
This also assumes that Seqno has a data type of int; otherwise you're likely to get a conversion error.
My guess is that the null reference exception occures on the code and has nothing to do with the sql query. It might just be that your code is not handling that you return no rows (or no scalar in your case) but you might be trying to access it somewhere in c#.
Show us the line of code that is throwing this exception in c# so we might be able to confirm this.
regards
Edit : From this similar topic
In your c# code you might want to try ("cmd" being your "SqlCommand" object):
int result = 0;
int.TryParse(cmd.ExecuteScalar(), out result);
or in one line
int.TryParse(cmd.ExecuteScalar(), out int result);
I don't know if it is the most suitable solution for you but I hope it is a start.
As covered null and no row are not the same.
This sample covers it.
set nocount on;
select isnull(1, 0)
where 1 = 1;
select isnull(1, 0)
where 1 = 2;
select isnull(null, 0)
where 1 = 1;
select isnull(null, 0)
where 1 = 2;
-----------
1
-----------
-----------
0
-----------
this should work
select top 1 isnull(seq, 0)
from (select null as seq
union all
select max(seq) from tblProjectChangeOrder where ProjectID = XXXXX
) tt
order by seq desc
LINQ to SQL is great however it backfires in performance in some cases. And with my case it backfires bigtime.
I have a StudentServiceDm that maps to a StudentService Table with 40 some fields. I have 10-20 millions of services in the table. The requirement from the client is that they need to export all services into excel from the application, while selecting only the fields they want at the time.
From my understanding, if i do
IQueryable<StudentServiceDm> serviceQuery = GetListQuery();
List<StudentServiceDto> dtos = serviceQuery.Select(m => new StudentServiceDto
{
Id = m.Id,
Name = m.Name,
}).ToList();
Then this translates into the SQL that selects only the fields I need
SELECT Id, Name FROM StudentService
Which saves performance by alot by decreasing the number of fields that need to be selected.
But in my case I need to allow the user to conditionally select fields like this, so I do this:
IQueryable<StudentServiceDm> serviceQuery = GetListQuery();
List<StudentServiceDto> dtos = serviceQuery.Select(m => new StudentServiceDto
{
Id = (options.ShowId) ? m.Id : null,
Name = (options.ShowName) ? m.Name,
StudentFirstName = (options.ShowFirstName) ? m.Student.FirstName : null,
StudentLastName = (options.ShowLastName) ? m.Student.LastName : null,
StudentServiceType = (options.ShowType) ? m.StudentServiceType.Name : null,
StudentServiceSubType = (options.ShowSubType) ? m.StudentServiceSubType.Name : null,
Date = (options.ShowDate) ? m.Date : null,
// alot more assignments .....
}).ToList();
However, this translates into the SQL query to something like this (according to SQL Profiler:
SELECT
CASE WHEN (#p__linq__1 = 1) THEN [Project2].[Date] END AS [C1],
CASE WHEN (#p__linq__2 = 1) THEN [Project2].[Name] END AS [C2],
CASE WHEN (#p__linq__3 = 1) THEN [Project2].[StudentServiceTypeId] END AS [C3],
CASE WHEN (#p__linq__4 = 1) THEN [Project2].[StudentServiceSubTypeId] END AS [C4],
CASE WHEN (#p__linq__5 = 1) THEN [Project2].[Date] END AS [C5],
// ..... more fields
FROM * //
where it puts a condition on every field to check if it should retrieve ... and actually this query's performance is almost the same as if all fields were selected.
What I want to achieve is, if the user only chooses to select Name and Date, then the translated SQL should only be
SELECT Name, Date FROM StudentService
After some performance analysis, the performance difference between these 2 sets of statement is about 3-4 times.
Any expert opinion?
You can create Linq expressions at runtime that exactly correspond to what you'd type in inside a .Select() statement, it's just not very straightforward. I never did it for Linq to SQL specifically but I don't see why it wouldn't work. Something like How do I dynamically create an Expression<Func<MyClass, bool>> predicate? but of course I'd recommend reading the docs first to get familiar with expressions.
The only way to do what you are interested in doing is to create lots of different Select() sections, each corresponding to a different combination of fields. However, you will need to create an awful lot of different combinations (N!, where N is the number of fields), so it is not really practical or maintainable.
There is no other way to do this using L2S.
If performance is important for you in this aspect, I suggest creating the sql directly and using something like Dapper to retrieve the data. Then you can easily control exactly the fields that you want to include.
I use asp.net 4 c# and ef4.
I have this code, it should compile a query and return a single scalar value (I use anonymous type).
My code does not have apparently errors, but because is the first time I write a compiled query I would like to know if is well written or could be improved for a performance boost.
var query = CompiledQuery.Compile((CmsConnectionStringEntityDataModel ctx)
=> from o in ctx.CmsOptions
where o.OptionId == 7
select new
{
Value = o.Value
});
uxHtmlHead.Text = query(context).FirstOrDefault().Value;// I print the result in a Label
SQL Profile Output:
SELECT TOP (1)
[Extent1].[OptionId] AS [OptionId],
[Extent1].[Value] AS [Value]
FROM [dbo].[CmsOptions] AS [Extent1]
WHERE 7 = [Extent1].[OptionId]
Many Thanks
Result after Wouter advice (please guys have a double check again):
static readonly Func<CmsConnectionStringEntityDataModel, int, string> compiledQueryHtmlHead =
CompiledQuery.Compile<CmsConnectionStringEntityDataModel, int, string>(
(ctx, id) => ctx.CmsOptions.FirstOrDefault(o => o.OptionId == id).Value);
using (var context = new CmsConnectionStringEntityDataModel())
{
int id = 7;
uxHtmlHead.Text = compiledQueryHtmlHead.Invoke(context, id);
}
Resulting SQL (I do not understand why with a LEFT JOIN)
exec sp_executesql N'SELECT
[Project1].[Value] AS [Value]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
LEFT OUTER JOIN (SELECT
[Extent1].[Value] AS [Value]
FROM [dbo].[CmsOptions] AS [Extent1]
WHERE [Extent1].[OptionId] = #p__linq__0 ) AS [Project1] ON 1 = 1',N'#p__linq__0 int',#p__linq__0=7
There are 2 things you can improve on.
First, precompiling a query is definitely a good idea but if you have a look at your code you will see that it precompiles the query each and every time instead of only once.
You need to move the precompiled query to a static variable that is initialized only once.
Another thing you need to be careful of is that when precompiling a query you shouldn't modify the query anymore before executing it.
You are building a precompiled query that will select all rows and then you say 'firstordefault' which changes the precompiled query to a SELECT TOP (1) and you lose the benefit of precompiling. You need to move the FirstOrDefault part inside your precompiled query and return only one result.
Have a look at this documentation. If you look at the examples you can see how they use a static field to hold the compiled query and how they specify the return value.
Scenario
I have a stored procedure that takes a single parameter. I want to update this stored procedure to take a VARIABLE NUMBER OF PARAMETERS - a number that I will never know.
I currently use SQLConnections through a C# interface in order to pass in a single parameter to the stored procedure and return a result.
The SQL Part
Lets say that I have a stored procedure that returns a list of results based on a single input parameter "#ccy" - (Currency). Now lets say that I want to update this stored procedure to take a list of Currencies instead of a single one, but that this number will be variable depending on the situation.
The SQL Code
ALTER PROCEDURE [dbo].[SEL_BootStrapperInstRICs]
(
#ccy varchar(10)
)
AS
SELECT DISTINCT i.CCY, i.Instrument, i.Tenor, r.RIC, r.[Server], r.RIType
FROM MDR.dbo.tblBootStrapperInstruments as i, MDR.dbo.tblBootStrapperRICs as r
WHERE i.Instrument = r.MurexInstrument
AND
i.Tenor = r.Tenor
AND i.CCY = r.CCY
AND i.CCY = #ccy
AND r.RIType NOT LIKE '%forward%'
The C# Part
This particular stored procedure is called from a C# WinForms application that uses the "SqlCommand.Parameters.AddWithValue()" method. As mentioned earlier this method currently passes in a single Currency as the parameter to the stored procedure and returns the result as a DataSet.
public DataSet GetBootStrapperInstRICsDS(List<string> ccys)
{
DataSet ds;
SqlConnection dbConn = null;
SqlCommand dbCmd = new SqlCommand();
try
{
dbConn = GetSQLConnection();
dbCmd = GetSqlCommand();
dbCmd.CommandType = CommandType.StoredProcedure;
dbCmd.CommandText = Utils.Instance.GetSetting ("SELBootStrapInsRics", "default");
foreach(string ccy in ccys)
dbCmd.Parameters.AddWithValue("#ccy", ccy);
dbCmd.CommandTimeout = 600;
dbCmd.Connection = dbConn;
SqlDataAdapter adapter = new SqlDataAdapter(dbCmd);
ds = new DataSet();
adapter.Fill(ds, "tblBootStrapperInstRICs");
dbCmd.Connection.Open();
return ds;
}
catch (Exception ex)
{
ApplicationException aex = new ApplicationException ("GetBootStrapperInstRICsDS", ex);
aex.Source = "Dal.GetBootStrapperInstRICsDS " + ex.Message;
MainForm.job.Log(aex.Source, Job.MessageType.Error);
Job.incurredErrors = true;
throw aex;
}
finally
{
if (dbCmd != null)
dbCmd.Dispose();
if (dbConn != null)
{
dbConn.Close();
dbConn.Dispose();
}
}
}
The Question
On the C# side I think my best option is to use a "foreach/for loop" in order to iterate through a list of parameters and dynamically add a new one to the SPROC. (I have already made this update in the C# code above).
HOWEVER - Is there some way that I can do this in the SQL Stored Procedure too? My thoughts are split with two potential options - Either create 20 or more parameters in the SPROC (each with the same name but with an incrementing number at the end e.g. - #ccy1,#ccy2 etc.) and use "for(int i=0;i
for(int i=0;i<NumberOfCurrenciesToAdd;i++)
dbCmd.Parameters.AddWithValue("#ccy"+i, currencyArray[i]);
Or the other option is to do something completely different and less rubbish and hack-esque. Help greatly appreciated.
EDIT - SQL Server 2005
EDIT2 - Must Use SPROCS - Company Specification Requirement.
You never specified SQL Server version, but for 2008 there are Table-Valued Parameters, which may help you:
Table-valued parameters are a new parameter type in SQL Server 2008. Table-valued parameters are declared by using user-defined table types. You can use table-valued parameters to send multiple rows of data to a Transact-SQL statement or a routine, such as a stored procedure or function, without creating a temporary table or many parameters.
I worked for a company that had to do this. It is much easier to just pass an nvarchar that is really a list that is comma delimited and then parse it when you get into the stored proc and insert the values into a temp table. The other option would be to have an xml parameter in your proc. That should also work. This is all for SQL 2005. 2008 does give you the table variable and that would be your best option.
I would try to stay away from dynamically changing your stored proc because I think that would be hard to maintain. At any given time if you try to look at the proc it could be different. Also, what happens when 2 people are trying to use your site and hit that proc at the same moment? One person's session will be modifying the procedure and the others will try to do it. This could cause a lock on the stored proc or it could cause other issues. Regardless it would be pretty inefficient.
Here is another option - though I think Anton's answer is better. You can pass in a csv string as a single parameter. Use a user-defined function to convert the csv string into a table of values, which you can join in your query. There are several csv parsing functions listed on SO and other places (though, sorry, I can't come up with a link right now).
edit: here is another option. Pass in the same csv string, then generate the sql query as a string in the procedure, and execute the string. Use the csv in an 'in' clause :
where i.ccy in (1,2,3,4)
I would not try to change the stored procedure, but (since you are on SQL Server 2005 and don't have table variable parameters) just pass in a comma separated list of values and let the procedure split them apart. You can change your C# loop to just build a CSV string and once you create a SQL split procedure, use it like:
SELECT
*
FROM YourTable y
INNER JOIN dbo.yourSplitFunction(#Parameter) s ON y.ID=s.Value
I prefer the number table approach to split a string in TSQL
For this method to work, you need to do this one time table setup:
SELECT TOP 10000 IDENTITY(int,1,1) AS Number
INTO Numbers
FROM sys.objects s1
CROSS JOIN sys.objects s2
ALTER TABLE Numbers ADD CONSTRAINT PK_Numbers PRIMARY KEY CLUSTERED (Number)
Once the Numbers table is set up, create this split function:
CREATE FUNCTION [dbo].[FN_ListToTable]
(
#SplitOn char(1) --REQUIRED, the character to split the #List string on
,#List varchar(8000)--REQUIRED, the list to split apart
)
RETURNS TABLE
AS
RETURN
(
----------------
--SINGLE QUERY-- --this will not return empty rows
----------------
SELECT
ListValue
FROM (SELECT
LTRIM(RTRIM(SUBSTRING(List2, number+1, CHARINDEX(#SplitOn, List2, number+1)-number - 1))) AS ListValue
FROM (
SELECT #SplitOn + #List + #SplitOn AS List2
) AS dt
INNER JOIN Numbers n ON n.Number < LEN(dt.List2)
WHERE SUBSTRING(List2, number, 1) = #SplitOn
) dt2
WHERE ListValue IS NOT NULL AND ListValue!=''
);
GO
You can now easily split a CSV string into a table and join on it:
select * from dbo.FN_ListToTable(',','1,2,3,,,4,5,6777,,,')
OUTPUT:
ListValue
-----------------------
1
2
3
4
5
6777
(6 row(s) affected)
Your can pass in a CSV string into a procedure and process only rows for the given IDs:
SELECT
y.*
FROM YourTable y
INNER JOIN dbo.FN_ListToTable(',',#GivenCSV) s ON y.ID=s.ListValue
I use this function to split CSV text into a table of numbers, it has great performance due to various optimizations (like returning a table with a primary key which greatly influence the query optimizer to produce good query plans ever for extremely large data sets).
Also it's not limited to 4000 characters, so you can pass in very large strings.
CREATE Function [dbo].[TextSplitToInt](#list text,
#delim char(1) = N',')
RETURNS #T TABLE (ID_T int primary key)
BEGIN
DECLARE #slices TABLE (slice nvarchar(4000) NOT NULL)
DECLARE #slice nvarchar(4000),
#textpos int,
#maxlen int,
#stoppos int
SELECT #textpos = 1, #maxlen = 4000 - 2
WHILE datalength(#list) / 2 - (#textpos - 1) >= #maxlen
BEGIN
SELECT #slice = substring(#list, #textpos, #maxlen)
SELECT #stoppos = #maxlen - charindex(#delim, reverse(#slice))
INSERT #slices (slice) VALUES (#delim + left(#slice, #stoppos) + #delim)
SELECT #textpos = #textpos - 1 + #stoppos + 2 -- On the other side of the comma.
END
INSERT #slices (slice)
VALUES (#delim + substring(#list, #textpos, #maxlen) + #delim)
INSERT #T (ID_T)
SELECT distinct Cast(str as int)
FROM (SELECT str = ltrim(rtrim(substring(s.slice, N.Number + 1,
charindex(#delim, s.slice, N.Number + 1) - N.Number - 1)))
FROM Numbers N
JOIN #slices s ON N.Number <= len(s.slice) - 1
AND substring(s.slice, N.Number, 1) = #delim) AS x
RETURN
END