I'm using Dapper to work with sql database.
I have some search logic in my website project.
My search gets list of string parameters.
//filter is list of strings
var sql = new StringBuilder();
sql.Append("SELECT LibraryDocumentId FROM LibraryDocumentKeywords WHERE ");
sql.Append(string.Join("OR ", filter.Select(f => string.Format("LOWER(Keyword) LIKE '%{0}%'", f)).ToList()));
var isList = conn.Query<int>(sql.ToString()).ToList();
Actually I don't want to use this approach of generating dynamic SQL query, because Dapper will cache every single query. I would prefer to pass filter with parameter. Can someone help me with that? Any idea ?
What you have at the moment is also a huge SQL injection risk. You might want to use DynamicParameters here, i.e. (completely untested, you may need to tweak slightly):
var sql = new StringBuilder(
"SELECT LibraryDocumentId FROM LibraryDocumentKeywords");
int i = 0;
var args = new DynamicParameters();
foreach(var f in filter) {
sql.Append(i == 0 ? " WHERE " : " OR ")
.Append("LOWER(Keyword) LIKE #p").Append(i);
args.Add("p" + i, "%" + f + "%");
i++;
}
var data = conn.Query<int>(sql.ToString(), args);
This should cache fairly cleanly (one cache item per number of filters, regardless of their contents).
Related
I have a list of items with different ids which represent a SQL table's PK values.
Is there any way to build an efficient and safe statement?
Since now I've always prepared a string representing the statement and build it as I traversed the list via a foreach loop.
Here's an example of what I'm doing:
string update = "UPDATE table SET column = 0 WHERE";
foreach (Line l in list)
{
update += " id = " + l.Id + " OR";
}
// To remove last OR
update.Remove(update.Length - 3);
MySqlHelper.ExecuteNonQuery("myConnectionString", update);
Which feels very unsafe and looks very ugly.
Is there a better way for this?
So yeah, in SQL you've got the 'IN' keyword which allows you to specify a set of values.
This should accomplish what you would like (syntax might be iffy, but the idea is there)
var ids = string.Join(',', list.Select(x => x.Id))
string update = $"UPDATE table SET column = 0 WHERE id IN ({ids})";
MySqlHelper.ExecuteNonQuery("myConnectionString", update);
However, the way you're performing your SQL can be considered dangerous (you should be fine as this just looks like ids from a DB, who knows, better to be safe than sorry). Here you're passing parameters straight into your query string, which is a potential risk to SQL injection which is very dangerous. There are ways around this, and using the inbuilt .NET 'SqlCommand' object
https://www.w3schools.com/sql/sql_injection.asp
https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlcommand?view=dotnet-plat-ext-6.0
It would be more efficient to use IN operator:
string update = "UPDATE table SET column = 0 WHERE id IN (";
foreach (Line l in list)
{
update += l.Id + ",";
}
// To remove last comma
update.Remove(update.Length - 1);
// To insert closing bracket
update += ")";
If using .NET Core Framework, see the following library which creates parameters for a WHERE IN. The library is a port from VB.NET which I wrote in Framework 4.7 years ago. Clone the repository, get SqlCoreUtilityLibrary project for creating statements.
Setup.
public void UpdateExample()
{
var identifiers = new List<int>() { 1, 3,20, 2, 45 };
var (actual, exposed) = DataOperations.UpdateExample(
"UPDATE table SET column = 0 WHERE id IN", identifiers);
Console.WriteLine(actual);
Console.WriteLine(exposed);
}
Just enough code to create the parameterizing SQL statement. Note ActualCommandText method is included for development, not for production as it reveals actual values for parameters.
public static (string actual, string exposed) UpdateExample(string commandText, List<int> identifiers)
{
using var cn = new SqlConnection() { ConnectionString = GetSqlConnection() };
using var cmd = new SqlCommand() { Connection = cn };
cmd.CommandText = SqlWhereInParamBuilder.BuildInClause(commandText + " ({0})", "p", identifiers);
cmd.AddParamsToCommand("p", identifiers);
return (cmd.CommandText, cmd.ActualCommandText());
}
For a real app all code would be done in the method above rather than returning the two strings.
Results
UPDATE table SET column = 0 WHERE id IN (#p0,#p1,#p2,#p3,#p4)
UPDATE table SET column = 0 WHERE id IN (1,3,20,2,45)
I have a list of keywords that i store in a list.
To fetch records from a table, am using the following query:
sqlBuilder.Append("SELECT name, memberid FROM members WHERE");
StringBuilder sqlBuilder = new StringBuilder();
foreach (string item in keywords)
{
sqlBuilder.AppendFormat(" LOWER(Name) LIKE '%{0}%' AND", item);
}
string sql = sqlBuilder.ToString();
As you might have noticed, my query is vulnerable to sql injection, thus i want to use parameters using SqlCommand(). I have tried the following but still doesn't work:
foreach (string item in keywords)
{
sqlBuilder.AppendFormat(" LOWER(Name) LIKE '%' + #searchitem + '%' AND", item);
SqlCommand cmd = new SqlCommand(sqlBuilder.ToString());
cmd.Parameters.AddWithValue("#searchitem",item);
}
Where could i be making the mistake, or rather, how should i got about it?
You are doing a few things wrong here:
You give all your parameters the same name #searchitem. That won't work. The parameters need unique names.
You create a new SqlCommand for each item. That won't work. Create the SqlCommand once at the beginning of the loop and then set CommandText once you are done creating the SQL.
Your SQL ends with AND, which is not valid syntax.
Improvement suggestions (not wrong per se, but not best practice either):
As Frederik suggested, the usual way is to put the % tokens in the parameter, rather than doing string concatenation inside the SQL.
Unless you explicitly use a case-sensitive collation for your database, comparisons should be case-insensitive. Thus, you might not need the LOWER.
Code example:
SqlCommand cmd = new SqlCommand();
StringBuilder sqlBuilder = new StringBuilder();
sqlBuilder.Append("SELECT name, memberid FROM members ");
var i = 1;
foreach (string item in keywords)
{
sqlBuilder.Append(i == 1 ? " WHERE " : " AND ");
var paramName = "#searchitem" + i.ToString();
sqlBuilder.AppendFormat(" Name LIKE {0} ", paramName);
cmd.Parameters.AddWithValue(paramName, "%" + item + "%");
i++;
}
cmd.CommandText = sqlBuilder.ToString();
Do not put the wildcard characters in your querystring, but add them to your parameter-value:
sql = "SELECT name FROM members WHERE Name LIKE #p_name";
...
cmd.Parameters.AddWithValue("#p_name", "%" + item + "%");
When you add the wildcard characters inside your query-string, the parameter will be escaped, but the wildcard chars will not; that will result in a query that is sent to the DB that looks like this:
SELECT name FROM members WHERE Name LIKE %'somename'%
which is obviously not correct.
Next to that, you're creating a SqlCommand in a loop which is not necessary. Also, you're creating parameters with a non-unique name, since you're adding them in a loop, and the parameter always has the same name.
You also need to remove the last AND keyword, when you exit the loop.
Here is a broken down version of my problem :
I have a method. The argument is a list of integers. I wish to create an optimised SQL query that returns a particular value on all rows where the id of that row is equal to one of the integers in the argument list. Simple, right ?
I have a feeling I have made it more difficult than it needs to be :
private List<string> ReturnValue(List<int> ids)
{
List<string> ValuesIWantToReturn = new List<string>();
StringBuilder sb = new StringBuilder("SELECT ValueIWantToReturnfrom table WHERE ");
foreach (int id in ids)
{
sb.Append( string.Format("ID = {0} OR ", id) );
}
sb.Remove(sb.Length - 3, 3); //remove trailing "OR"
sb.Append(";");
SqlDataReader reader = RunSelectQuery( sb.ToString() );
while (reader.Read())
{
ValuesIWantToReturn.Add(reader.GetString(0));
}
return ValuesIWantToReturn;
}
Any feedback on the general readability and structure of my code would be appreciated too. It's always nice to improve :)
You can use the IN syntax rather than OR
WHERE id IN (1,2,3,4)
It may be more efficient to pass the list of IDs as a table valued parameter to a stored procedure, and then join that to your table, but that may be over engineering the solution - depending on how long your list of numbers is.
Instead of your loop, you can use string.Join to build your list
string query = "SELECT ValueIWantToReturn from table WHERE ID IN ("
+ string.Join(",", ids)
+ ")";
(assuming that there is always at least one number in the list)
I have an application I need to create which, given some user input in the form of CSV, needs to parse and generate this CSV into multiple formats. One of these formats is a series of SQL INSERT statements (as a string) for each line of CSV.
(At this point you can assume I've already parsed the CSV into a list of values or something, so that is not the point of the question)
Given that this input could contain vulnerabilities, I wish to generate the INSERT statements which have been validated and sanitised.
I am familiar with creating an SqlCommand object and adding values to its list of Parameters, but looking over a similar question it doesn't appear to work in way I had hoped.
So is there a way to generate sanitised SQL statements, as strings, in the way I need to?
EDIT: This is an example what I want to do.
CSV:
id,name,country
1,Alice,France
2,Bob,Austria
3,Carol,Germany
SQL:
...
INSERT INTO Users (id, name, country) VALUES (1, 'Alice', 'France');
INSERT INTO Users (id, name, country) VALUES (2, 'Bob', 'Austria');
INSERT INTO Users (id, name, country) VALUES (3, 'Carol', 'Germany');
...
As there are no data types given in the CSV, the application has to determine that as well.
Not so much an answer, as a cautionary note. If you end up needing to go the 'classic' escaping route to do this, and really need safety (i.e. the data is coming in from untrusted source), don't forget it's not just simple escaping you need to worry about.
Basic character escaping we hear about all the time:
' -> '' apostrophe's and stuff are quite obvious and documented ad-nauseum
; multiple-commands in one statement - not always allowed by the DB, but dangerous
If you're parsing for "nefarious behaviour" though, have you thought about:
SELECT/*not important*/1/*really...*/FROM/*im serious*/users
SELECT%09FIELDS%09FROM%0dTABLE_NAME
WHERE username=CONCAT(CHAR(97),CHAR(100),CHAR(109),CHAR(105),CHAR(110))
SELECT passwd FROM users WHERE username=0x61646d696e
In summary: Here Be Dragons.
http://www.ihteam.net/papers/blind-sqli-regexp-attack.pdf
http://ferruh.mavituna.com/sql-injection-cheatsheet-oku/#HexbasedSamples
Well chances are if you don't wan't to use SQL objects then you would have to sanatize the entries yourself. I'm not aware of any recommended format for SQL however for MySQL the following would work. I've changed it to work with SQL however I cant garantee it has covered all possible injection attacks.
public string sqlEscape(string VAL)
{
if (VAL == null)
{
return null;
}
return "\"" + Regex.Replace(VAL, #"[\r\n\x00\x1a\\'""]", #"\$0") + "\"";
}
to use you would then do (assuming your CSV line is stored in an array called csv):
string query = #"INSERT INTO Users (id, name, country) VALUES (" + sqlEscape(csv[0]) + ", " + sqlEscape(csv[1]) + ", " + sqlEscape(csv[2]) + ");";
if anyone can enhance this let me know!
Because i don't know how you've stored your variables, i'll show you a complete, possible implementation with your sample data using a List<Dictionary<String, Object>>():
Add your sample-data:
var tableName = "Users";
var records = new List<Dictionary<String, Object>>();
var recordFields = new Dictionary<String, Object>();
recordFields.Add("id", 1);
recordFields.Add("name", "Alice");
recordFields.Add("country", "France");
records.Add(recordFields);
recordFields = new Dictionary<String, Object>();
recordFields.Add("id", 2);
recordFields.Add("name", "Bob");
recordFields.Add("country", "Austria");
records.Add(recordFields);
recordFields = new Dictionary<String, Object>();
recordFields.Add("id", 3);
recordFields.Add("name", "Carol");
recordFields.Add("country", "Germany");
records.Add(recordFields);
Generate the parametrized insert statements:
using (var con = new SqlConnection(Settings.Default.ConnectionString))
{
con.Open();
foreach (var record in records)
{
String insertSql = String.Format("INSERT INTO {0} ({1}) VALUES ({2});"
, tableName
, String.Join(",", record.Select(r => r.Key))
, String.Join(",", record.Select(r => "#" + r.Key)));
using (var insertCommand = new SqlCommand(insertSql, con))
{
foreach (var field in record)
{
var param = new SqlParameter("#" + field.Key, field.Value);
insertCommand.Parameters.Add(param);
}
insertCommand.ExecuteNonQuery();
}
}
}
Note that this is not really tested(it compiles and looks good) but it should help you anyway.
Edit:
#Oded: I think the problem is that daniel doesn't want to run the
inserts, but show/output/save them. So using the SqlCommand parameters
is no good, because you don't see the generated SQL.
#Cylindric That's correct
That's not possible and a contradiciton, you cannot use a String with SqlParameters. So i'm afraid you're open to Sql-Injection if you would run these inserts later. I would suggest to use above code when you're actually running the statemenets.
So I have my SqlDataSource with a SelectQuery defined as follows:
SELECT * FROM table
WHERE UserName IN (#EmployeesIn);
With #EmployeesIn coming from a session variable Session["EmployeesIn"]. During Page_Load I'm taking an ArrayList members and putting the results into a string and setting the session variable:
string employeesIn = "";
foreach (string s in members)
{
employeesIn = employeesIn + "'" + s + "',";
}
employeesIn = employeesIn.TrimEnd(',');
Session["EmployeesIn"] = employeesIn;
Writing the output to the console I can see the value of the parameter #EmployeesIn
#EmployeesIn = 'bob', 'joe'
However, I'm getting zero results back ... and after monitoring from the database level I see the parameter is coming in as:
'''bob'',''joe'''
Then again if I just pass in one employee, I get results back from the SQL as expected and the parameter is passed correctly as just 'bob'. I suppose this is some safety that .NET provides against SQL injection attacks, however what's the safe way around this?
You should absolutely use parameters for this, instead of including the values within the SQL itself. You can just generate the names for the parameters, so if you had three entries you'd generate SQL of:
SELECT * FROM table WHERE UserName IN (#p0, #p1, #p2)
and then fill in those three parameters from the three values.
// Or create the command earlier, of course
List<SqlParameter> parameters = new List<SqlParameter>();
StringBuilder inClause = new StringBuilder("(");
for (int i = 0; i < members.Count; i++)
{
string parameterName = "#p" + i;
inClause.Append(parameterName);
if (i != members.Count - 1)
{
inClause.Append(", ");
}
// Adjust data type as per schema
SqlParameter parameter = new SqlParameter(parameterName, SqlDbType.NVarChar);
parameter.Value = members[i];
parameters.Add(parameter);
}
inClause.Append(")");
// Now use inClause in the SQL, and parameters in the command parameters
I think you have three options:
Comma separated values - you can pass single parameter value as CSVs and split them out in the stored procedure. I don't like this idea ... too many limitations.
XML - you can pass an XML string into the stored procedure and open it as a table using OPENXML. This will give you a table that you can use to do joins, inserts, etc., onto other tables.
Table-Valued Parameters
The better way would be to user your members array to build the query using a parameter list:
SELECT * FROM table
WHERE UserName IN (#EmployeesIn1, #EmployeesIn2, #EmployeesIn3, ... n);
Then loop through your member list, adding the parameters as necessary.
The first thing I noticed was how you're making you're comma demilited list. Try this out:
string employeesIn = string.Join(",", members.Select(x => string.format("'{0}'", x)).ToArray());
As for the parameter, you need to rethink your approach. Have you looked at table value parameters?
SQL Parameters can only represent a single value, you can however pass in multiple parameters such as:
var emps = members.Select( (e,i) => "#EMPLOYEE" + i).ToArray();
string sqlQuery = "SELECT * FROM table WHERE ";
sqlQuery+=string.Format("UserName IN ({0})", string.Join(",", emps));
//add SQL parameters used in query
for (int i = 0; i < members.Count; ++i)
{
parameters.Parameters.Add(new SqlParameter("#EMPLOYEE" + i, members[i]));
}