My ASP.NET MVC 4 application uses MS-SQL user defined functions to do a fulltext search.
I followed this post and created following code:
in Model Class:
if (suchstring.Trim() != "")
{
//search for each piece separated by space:
var such = suchstring.Split(' ');
int index = 0;
foreach (string teil in such)
{
index++;
if (teil.Trim() != "")
{
res = res.Join(db.udf_FirmenSucheMultiple(string.Format("\"{0}*\"", teil), index), l => l.ID, s => s.KEY, (l, s) => l);
}
}
}
Mapping function:
[EdmFunction("TQCRMEntities", "udf_AnsprechpartnerFirmaSuche")]
public virtual IQueryable<udf_AnsprechpartnerFirmaSuche_Result> udf_AnsprechpartnerFirmaSucheMultiple(string keywords, int index)
{
string param_name = String.Format("k_{0}", index);
var keywordsParameter = keywords != null ?
new ObjectParameter(param_name, keywords) :
new ObjectParameter(param_name, typeof(string));
return ((IObjectContextAdapter)this).
ObjectContext.CreateQuery<udf_AnsprechpartnerFirmaSuche_Result>(
String.Format("[TQCRMEntities].[udf_AnsprechpartnerFirmaSuche](#{0})", param_name), keywordsParameter);
}
SQL User defined function:
create function udf_AnsprechpartnerFirmaSuche
(#keywords nvarchar(4000))
returns table
as
return (select [KEY], [rank] from containstable(AnsprechpartnerFirma, *, #keywords, LANGUAGE 1031))
If I try to search for " I get a 500 Server Error (Syntaxerror from the SQLServer).
My question is if my app is vulnerable to SQL injections and how I should protect against them.
Is it save to just remove * and " from the input?
From http://msdn.microsoft.com/en-us/library/ms189760.aspx
CONTAINSTABLE is used in the FROM clause of a Transact-SQL SELECT statement and is referenced as if it were a regular table name. .It performs a SQL Server full-text search on full-text indexed columns containing character-based data types.
If you read Quassnoi's answer with regard to searching the full-text index for double quotes:
Punctuation is ignored. Therefore, CONTAINS(testing, "computer failure") matches a row with the value, "Where is my computer? Failure to find it would be expensive."
Documentation can be found here.
See his answer for an alternative using the LIKE operator.
To answer your questions:
My question is if my app is vulnerable to SQL injections and how I should protect against them.
You are using parameters properly in your UDF. It should be safe from SQL injection.
Is it save to just remove * and " from the input?
No. Never try to blacklist characters in an attempt to prevent SQL injection. You will almost certainly fail.
See OWASP SQL Injection Prevention for details.
Related
I have the following piece of code which is working fine:
ObjectContext octx = new ObjectContext("name=PublisherModelContainer");
ObjectSet<Author> authorSet = octx.CreateObjectSet<Author>();
ObjectQuery<Author> q = authorSet.Where("it.FirstName == #FirstName", new ObjectParameter("FirstName", "Isaak"));
Author a = q.FirstOrDefault();
if (a == null)
{
Console.WriteLine("Author not found");
return;
}
Console.WriteLine("{0} {1}", a.FirstName, a.LastName);
While calling the 'Where' method, the FirstName property is being referenced via "it.FirstName". What does this mean? I have tried using a different alias e.g. "a.FirstName" but that fails with exception message 'a.FirstName' could not be resolved in the current scope or context.
Even in Microsoft's example here (https://msdn.microsoft.com/en-us/library/bb338811%28v=vs.110%29.aspx), it.ProductID is being used not something like t.ProductID.
What exactly is "it"? Is it that "it" has a special meaning?
It is just how you can refer to the set you're currently working with. Same as with this in C# class. But this is achieved based on the query that EF generates.
Consider the following SQL script.
SELECT Col1, Col2, Col3 FROM Table AS It
WHERE It.Col1 = #param1
Here is a link with more detailed explanation http://www.w3schools.com/sql/sql_alias.asp
I create a complex search query in native SQL. It's basically something like this:
SELECT ID FROM t_Product WHERE Name LIKE #criteria
SELECT publisher, count(*) as number FROM t_Product GROUP BY publisher
It has 2 SELECT statements and I want it to be sent to DB server in one round trip.
But I can't figure out how to achieve this in Nhibernate.
I considered following options but none seems to work
Use CreateMultiQuery, but this only accept HQL, not native SQL
Use CreateSQLQuery, but call to List() only return result for the first SELECT statement
Moving to a stored procedure is not an option since the whole SQL is very dynamic.
We still use Nhibernate 1.2 thus new features in later version couldn't be used either.
Advice are welcome.
Not possible using NH version 1.2
Futures was released in version 2.1 which allows you to do exactly this.
e.g.
var blogs = s.CreateCriteria<Invoice>()
.SetMaxResults(30)
.Future<Invoice>();
var countOfInvoices = s.CreateCriteria<Invoice>()
.SetProjection(Projections.Count(Projections.Id()))
.FutureValue<int>();
So you are going to either upgrade, fall back to ADO.NET and use multiple recordsets or live with what you have! Sorry!
This is really going to be scenario-specific, but if you're stuck with NH Version 1.2, and eliminating the round-trip is your goal, you could consider rewriting this as a single query using a sub-select.
Something along the lines of:
SELECT publisher, count(*) as number,
(SELECT ID FROM t_Product WHERE Name LIKE #criteria) As theId
FROM t_Product GROUP BY publisher
Would work if your subquery only returned a single value.
I don't think that it is possible, because both queries are SELECTs.
You may try a semicolon after the first query, and two line feeds between them, this is required for some databases. I successfully run query-scripts like this. If it runs, use a debugger to see what you get back ...
If this doesn't work, you need separate round trips or switch to HQL / Criteria.
You can use MultiQuery "Hack" like this:
The procudure:
CREATE PROCEDURE [dbo].[proc_Name]
AS BEGIN
SELECT * FROM t_Question where ...
SELECT * FROM t_Question where ........
END
The NHibernate Query Code:
public void ProcdureMultiTableQuery()
{
var session = Session;
var procSQLQuery = session.CreateSQLQuery("exec [proc_Name] ?,?");// prcodure returns two table
procSQLQuery.SetParameter(0, userId);
procSQLQuery.SetParameter(1, page);
procSQLQuery.AddEntity(typeof(Question));
var multiResults = session.CreateMultiQuery()
.Add(procSQLQuery)
// More table your procedure returns,more empty SQL query you should add
.Add(session.CreateSQLQuery(" ").AddEntity(typeof(Question))) // the second table returns Question Model
.List();
if (multiResults == null || multiResults.Count == 0)
{
return;
}
if (multiResults.Count != 2)
{
return;
}
var questions1 = ConvertObjectsToArray<Question>((System.Collections.IList)multiResults[0]);
var questions2 = ConvertObjectsToArray<Question>((System.Collections.IList)multiResults[1]);
}
static T[] ConvertObjectsToArray<T>(System.Collections.IList objects)
{
if (objects == null || objects.Count == 0)
{
return null;
}
var array = new T[objects.Count];
for (int i = 0; i < array.Length; i++)
{
array[i] = (T)objects[i];
}
return array;
}
trying to get my head around using Lambda expressions to fetch data from my database.
Say I have a table that looks a bit like this (notice the spaces and casing):
name, count:
iPhone 4, 15
iphone 4, 2
iPhone4, 8
If I try to find items by name (using StartsWith()), I only want to fetch the result with the highest count, independent of casing and spaces. So searches for "iphone4" "i p h o n e 4", "iPhone4" sholud all return the "iPhone 4"-record
If you have a MS Sql Server 2005+ the following would work for your stated example:
var inputString = "iPhone 4";
var token = inputString.ToLower().Replace(" ", "");
var tokenizedQuery = DataContext.Devices.Select(d => new { Device = d, Token = d.Name.ToLower().Replace(" ", "") });
var filteredQuery = tokenizedQuery.Where(d => d.Token == token);
var resultsQuery = filteredQuery.Select(d => d.Device).OrderByDescending(d => d.Count);
var result = resultsQuery.FirstOrDefault();
Here is what is going on:
You are creating a tokenized version of your input string by lower-casing it and then removing spaces.
Then you are creating a pseudo-column on your table to create a similar token column
Filter your results based on this token
Finally, select only the record with the highest count
However it is very important that you realize the ToLower() and Replace() methods are being translated to T-SQL commands that run on the sql server and not in your app. This means should you need more sophisticated tokenizing routines, or you are not using MS SQL this may not work!
As others have noted, you may want to clean up your design somewhat. You are essentially storing a key or search keyword that can have many permutations. Doing the tokenizing in a query is not portable or performant, so you should ideally store the tokenized version of this string in its own column. Alternatively, look into Full Text Indexes, as they may also address your problem (again, if using MSSQL).
Let's assume that you have a Collapse string extension, which is not hard to write. One thing you'll note is that there won't be a mapping from this to SQL so the final filtering will have to be done in LINQ to Objects. You might be able to make the DB query more efficient by doing partial filtering (i.e., on iphone), then complete the filtering in memory.
db.Table.ToList().Where( t => t.Name.Collapse().StartsWith( searchString.Collapse() )
.OrderByDescending( t => t.Count )
.Take( 1 );
Where Collapse is
public static class StringExtensions
{
public static string Collapse( this string source )
{
if (string.IsNullOrWhiteSpace( source ))
{
return string.Empty;
}
var builder = new StringBuilder();
foreach (char c in source)
{
if (!char.IsWhiteSpace( c ))
{
builder.Append( c );
}
}
return builder.ToString();
}
}
Note: you'd be better off sanitizing your database if possible AND you really want these to map to the same thing.
I have several methods that use similar linq statements but different enough for them to be in their own methods. So say, for the sake of arguemnt, I had the following linq snippet which is repeated across all methods (the real snippets would be much longer than this):
where su.ObjId == serviceUserId
where cl.StaffMemberId == staffMemberId
If I was working in SQL I could just contatenate the repeated SQL as follows:
private string GetRepeatedSql()
{
return "where su.ObjId = serviceUserId and cl.StaffMemberId = staffMemberId";
}
private void DoSomething()
{
string mySql = "Select * from ...... lots of sql .." + GetRepeatedSql() + ".. some more sql";
}
(Usual health warnings around contatenating SQL string together noted).
Is there something equivalent in Linq? I'm sick of having to make changes in several places - this seems to contravene the DRY principle.
Thanks!
Correct me if I'm wrong but I always thought LINQ statements weren't executed until you actually used them. (Coming from LINQ to NHibernate)
If that is actually the case you could simply just add whatever you need to the existing statement. For example:
var temp=from x in Sometable select x;
Then adding a where clause:
temp = from x in temp where x.ID==1234 select x;
Then order by
temp=from x in temp order by x.ID select x;
I won't lie I have never done it this way but I assume it should work. If someone knows this won't work please explain why. Thanks.
Found this on msdn: http://msdn.microsoft.com/en-us/library/bb397906.aspx
In LINQ the execution of the query is
distinct from the query itself; in
other words you have not retrieved any
data just by creating a query
variable.
So by creating the variable you have not retrieved any data. Although maybe the way I'm doing it above would return data because I am calling from x in temp to change the query.
I do it like this
IQueryable<Publication> pubs = GetPubs();
pubs = ApplySort(pubs, SortBy);
pubs = GetPage(pubs, PageSize, Page);
private IQueryable<Publication> GetPage(IQueryable<Publication> pubs, int PageSize, int Page)
{
return pubs.Skip(PageSize * (Page - 1)).Take(PageSize);
}
private IQueryable<Publication> ApplySort(IQueryable<Publication> pubs, string SortBy)
{
switch (SortBy)
{
case "Latest": return pubs.OrderByDescending(p => p.Posted);
break;
default: return pubs.OrderByDescending(p => p.Posted);
break;
}
}
You can use PredicateBuilder to do this:
The Albahari one here is one I've used recently although there are others around:
http://www.albahari.com/nutshell/predicatebuilder.aspx
Is there a "best practice" way of handling bulk inserts (via LINQ) but discard records that may already be in the table? Or I am going to have to either do a bulk insert into an import table then delete duplicates, or insert one record at a time?
08/26/2010 - EDIT #1:
I am looking at the Intersect and Except methods right now. I am gathering up data from separate sources, converting into a List, want to "compare" to the target DB then INSERT just the NEW records.
List<DTO.GatherACH> allACHes = new List<DTO.GatherACH>();
State.IState myState = null;
State.Factory factory = State.Factory.Instance;
foreach (DTO.Rule rule in Helpers.Config.Rules)
{
myState = factory.CreateState(rule.StateName);
List<DTO.GatherACH> stateACHes = myState.GatherACH();
allACHes.AddRange(stateACHes);
}
List<Model.ACH> newRecords = new List<Model.ACH>(); // Create a disconnected "record set"...
foreach (DTO.GatherACH record in allACHes)
{
var storeInfo = dbZach.StoreInfoes.Where(a => a.StoreCode == record.StoreCode && (a.TypeID == 2 || a.TypeID == 4)).FirstOrDefault();
Model.ACH insertACH = new Model.ACH
{
StoreInfoID = storeInfo.ID,
SourceDatabaseID = (byte)sourceDB.ID,
LoanID = (long)record.LoanID,
PaymentID = (long)record.PaymentID,
LastName = record.LastName,
FirstName = record.FirstName,
MICR = record.MICR,
Amount = (decimal)record.Amount,
CheckDate = record.CheckDate
};
newRecords.Add(insertACH);
}
The above code builds the newRecords list. Now, I am trying to get the records from this List that are not in the DB by comparing on the 3 field Unique Index:
AchExceptComparer myComparer = new AchExceptComparer();
var validRecords = dbZach.ACHes.Intersect(newRecords, myComparer).ToList();
The comparer looks like:
class AchExceptComparer : IEqualityComparer<Model.ACH>
{
public bool Equals(Model.ACH x, Model.ACH y)
{
return (x.LoanID == y.LoanID && x.PaymentID == y.PaymentID && x.SourceDatabaseID == y.SourceDatabaseID);
}
public int GetHashCode(Model.ACH obj)
{
return base.GetHashCode();
}
}
However, I am getting this error:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable1[MisterMoney.LARS.ZACH.Model.ACH] Intersect[ACH](System.Linq.IQueryable1[MisterMoney.LARS.ZACH.Model.ACH], System.Collections.Generic.IEnumerable1[MisterMoney.LARS.ZACH.Model.ACH], System.Collections.Generic.IEqualityComparer1[MisterMoney.LARS.ZACH.Model.ACH])' method, and this method cannot be translated into a store expression.
Any ideas? And yes, this is completely inline with the original question. :)
You can't do bulk inserts with LINQ to SQL (I presume you were referring to LINQ to SQL when you said "LINQ"). However, based on what you're describing, I'd recommend checking out the new MERGE operator of SQL Server 2008.
Inserting, Updating, and Deleting Data by Using MERGE
Another example here.
I recommend you just write the SQL yourself to do the inserting, I find it is a lot faster and you can get it to work exactly how you want it to. When I did something similar to this (just a one-off program) I just used a Dictionary to hold the ID's I had inserted already, to avoid duplicates.
I find LINQ to SQL is good for one record or a small set that does its entire lifespan in the LINQ to SQL.
Or you can try to use SQL Server 2008's Bulk Insert .
One thing to watch out for is if you queue more than 2000 or so records without calling SubmitChanges() - TSQL has a limit on the number of statements per execution, so you cannot simply queue up every record and then call SubmitChanges() as this will throw an SqlException, you need to periodically clear the queue to avoid this.