Consider the following scenario:
http://www.yourdomain.com/Default.aspx?p=2
Now we ofcourse want to check if the querystring parameter p doesnt contain errors.
I now have this setup:
1) Check if p exists
2) Filter out html from p's value
3) htmlencode p's value
4) check if p is integer
5) check if p's integer exists in db
This is how I usual do it, though step 5 is ofcourse a performance hit.
Kind regards,
Mark
My view: Generally a querystring parameter of this kind isn't really "entered" by users but is submitted as a link. So over-complex slow validation isn't really necessary.
So I would just pass this through to the persistence / data layer and handle any errors that come back as a regular 404 Not Found or 500 Internal Server Error depending on the kind of system I'm working with.
If your intent is to use the parameter to retrieve something from the database, why filter out html or encode it? It's not like you're going to store it in the database, or display it on the front end. Just immediately throw it to the DAL if it exists. You're DAL should be smart enough to tell you if it failed to retrieve a record with that ID, or if the ID couldn't be parsed, etc..
If you are going to convert the input to an integer anyway, then steps 2 and 3 are not needed - just use int.TryParse to see what you have. I would encode and test the input for html only if you are expecting a string which you will use in a dynamic sql statement, or will be displaying on your site
What about:
int p = 0;
if(!Int32.TryParse(Request.QueryString["p"], out p))
throw new ArgumentOutOfRangeException("p");
Quite simple. For most data types (integers, decimals, doubles, dates and booleans) there is a very strict format. If the value does not parse under the strict format, it's an error.
Strings sometimes have a strict format, like an email address or a phone number. Those can be validated with a simple regexp. If it conforms, use it, otherwise it's an error.
Most of the time however strings will simply need to be persisted to the DB and later displayed again. In that case no processing is needed, aside from escaping when inserting into DB (unnecessary as well if you used parametrized queries)k, and HTML-encoding when rendering to the display.
This way any and all data is validated, and there is no risk of any injections whatsoever.
The rare exception of a loose format for a string is, well... rare. I can't think of any right now. For that you can afford some more extensive parsing and processing.
Added: Oh, yes, checking whether IDs (or other values) are valid in respect to a DB. You're doing it right, but think if you always need it. Quite often you can put the check into some other query that you have to do anyway. Like when you select data based on the ID, you don't need to explicitly check that it exists - just be ready that your query can return no data.
Sometimes you don't need to use the value at all, then you can simply ignore it.
But, of course, there are other times, like when inserting/updating data, that you indeed need to explicitly check whether the data exists and is valid in the current context.
Related
I would like to add an option that user using web ui can define some views in our product.
We let super admins put in SQL query that would create that view.
Do you know if there are any tools for parsing SQL Query and check if it is valid SQL Query for only creating view?
I assume the input is secure, so your superusers are not going to try to inject stuff ;-)
Anyway the comment above is correct, at least you should check for semicolon and comments. Also that the string starts with "SELECT "
The easiest way - after this basic checks - would be to let them insert the SELECT-Query alone and test it on your connections. Then catch any exception and display it to the superuser.
In case of success, you could even display some of the result, in order give them the possibility to validate the query.
After that you prefix in your code the SELECT with "CREATE VIEW AS " in order to create the view on your database.
One possibility is to create a unique role that has select access on every table in the database but nothing else. Then have your C# run the query as that user in a try/catch block. This would even eliminate the possibility of SQL injection and/or provide a failsafe for well meaning but careless users.
The main disadvantage I see is that you would need to continually update permissions to ensure that the user is granted select access to new objects, but I think you can define those types of cascading permissions at the schema level... never tried it at the database level.
Force the query to begin with “CREATE VIEW”, e.g. by prefixing whatever is entered with that string.
Make sure that there is no semicolon inside the query. This will exclude some valid queries (that contain something like “WHERE a = ';'” or “/* comment; */”), but it is simple and you might get away with it.
Alright so I have a user table that I would like to check against. This user table has a username, email, and accountnumber. Would I would like to do is check if anyone of those has been taken and if it has return if it has been taken.
I was thinking about doing an array example
CHECK AGAINST TABLE
IF USERNAME MATCHES INSERT #ARRAY "TRUE"
IF EMAIL MATCHES INSERT #ARRAY "TRUE"
ETC.
Then on the C# side I will call the array and check by index
If registrationValidationArray[0] = "true"
{
ViewBag.UserNameTaken = "True"
return view("Registration")
// On cshtml post error next to username that it has been taken
}
So my question is does this approach sound logical and sound like it will work or is there another approach that might help me here procedure wise. Another developer suggested an incremental for the procedure so if username is taken +1 and then on the C# side display according to the numeric value but I couldn't wrap my head around that one. Anyone know of a better way or see a flaw in my logic?
If you are using the Telerik OpenAccess ORM and you have a stored procedure returning the details from the check I would suggest you to map this stored procedure to a Domain Method, then OpenAccess will return you an object with nicely named properties. You can use the returned object instead of the array. This will make you code more readable and easier to maintain. Now you should remember what means index 0 and index 1, even if you define the indexes as constants it will not be an eye-catching code.
This is a "best practice" question. We are having internal discussions on this topic and want to get input from a wider audience.
I need to store my data in a traditional MS SQL Server table with normal columns and rows. I sometimes need to return a DataTable to my web application, and other times I need to return a JSON string.
Currently, I return the table to the middle layer and parse it into a JSON string. This seems to work well for the most part, but does occasionally take a while on large datasets (parsing the data, not returning the table).
I am considering revising the stored procedures to selectively return a DataTable or a JSON string. I would simply add a #isJson bit parameter to the SP.
If the user wanted the string instead of the table the SP would execute a query like this:
DECLARE #result varchar(MAX)
SELECT #result = COALESCE(#results ',', '') + '{id:"' + colId + '",name:"' + colName + '"}'
FROM MyTable
SELECT #result
This produces something like the following:
{id:"1342",name:"row1"},{id:"3424",name:"row2"}
Of course, the user can also get the table by passing false to the #isJson parameter.
I want to be clear that the data storage isn't affected, nor are any of the existing views and other processes. This is a change to ONLY the results of some stored procedures.
My questions are:
Has anyone tried this in a large application? If so, what was the result?
What issues have you seen/would you expect with this approach?
Is there a better faster way to go from table to JSON in SQL Server other than modifying the stored procedure in this way or parsing the string in the middle tier?
I personally think the best place for this kind of string manipulation is in program code in a fully expressive language that has functions and can be compiled. Doing this in T-SQL is not good. Program code can have fast functions that do proper escaping.
Let's think about things a bit:
When you deploy new versions of the parts and pieces of your application, where is the best place for this functionality to be?
If you have to restore your database (and all its stored procedures) will that negatively affect anything? If you are deploying a new version of your web front end, will the JSON conversion being tied into the database cause problems?
How will you escape characters properly? Are you sending any dates through? What format will date strings be in and how will they get converted to actual Date objects on the other end (if that is needed)?
How will you unit test it (and with automated tests!) to prove it is working correctly? How will you regression test it?
SQL Server UDFs can be very slow. Are you content to use a slow function, or for speed hack into your SQL code things like Replace(Replace(Replace(Replace(Value, '\', '\\'), '"', '\"'), '''', '\'''), Char(13), '\n')? What about Unicode, \u and \x escaping? How about splitting '</script>' into '<' + '/script>'? (Maybe that doesn't apply, but maybe it does, depending on how you use your JSON.) Is your T-SQL procedure going to do all this, and be reusable for different recordsets, or will you rewrite it each time into each SP that you need to return JSON?
You may only have one SP that needs to return JSON. For now. Some day, you might have more. Then if you find a bug, you have to fix it in two places. Or five. Or more.
It may seem like you are making things more complicated by having the middle layer do the translation, but I promise you it is going to be better in the long run. What if your product scales out and starts going massively parallel—you can always throw more web servers at it cheaply, but you can't so easily fix database server resource saturation! So don't make the DB do more work than it should. It is a data access layer, not a presentation layer. Make it do the minimum amount of work possible. Write code for everything else. You will be glad you did.
Speed Tips for String Handling in a Web Application
Make sure your web string concatenation code doesn't suffer from Schlemiel the Painter's Algorithm. Either directly write to the output buffer as JSON is generated (Response.Write), or use a proper StringBuilder object, or write the parts of the JSON to an array and Join() it later. Don't do plain vanilla concatenation to a longer and longer string over and over.
Dereference objects as little as possible. I don't know your server-side language, but if it happens to be ASP Classic, don't use field names--either get a reference to each field in a variable or at the very least use integer field indexes. Dereferencing a field based on its name inside a loop is (much) worse performance.
Use pre-built libraries. Don't roll your own when you can use a tried and true library. Performance should be equal or better to your own and (most importantly) it will be tested and correct.
If you're going to spend the time doing this, make it abstract enough to handle converting any recordset, not just the one you have now.
Use compiled code. You can always get the fastest code when it is compiled, not interpreted. If you identify that the JSON-conversion routines are truly the bottleneck (and you MUST prove this for real, do not guess) then get the code into something that is compiled.
Reduce string lengths. This is not a big one, but if at all possible use one-letter json names instead of many-letter. For a giant recordset this will add up to savings on both ends.
Ensure it is GZipped. This is not so much a server-side improvement, but I couldn't mention JSON performance without being complete.
Passing Dates in JSON
What I recommend is to use a separate JSON schema (itself in JSON, defining the structure of the virtual recordset to follow). This schema can be sent as a header to the "recordset" to follow, or it can be already loaded in the page (included in the base javascript files) so it doesn't have to be sent each time. Then, in your JSON parse callback (or post-callback on the final resultant object) look in the schema for the current column and do conversions as necessary. You might consider using ISO format since in ECMAScript 5 strict mode there is supposed to be better date support and your code can be simplified without having to change the data format (and a simple object detect can let you use this code for any browser that supports it):
Date
Dates are now capable of both parsing and outputting ISO-formatted dates.
The Date constructor now attempts to parse the date as if it was ISO-formatted, first, then moves on to the other inputs that it accepts.
Additionally, date objects now have a new .toISOString() method that outputs the date in an ISO format.
var date = new Date("2009-05-21T16:06:05.000Z");
print( date.toISOString() );
// 2009-05-21T16:06:05.000Z
I wouldn't do that way you are doing (contatenating)
You can try creating a CLR SQL function that uses JSON.net and returns a varchar.
See here how to create SQL CLR Functions:
http://msdn.microsoft.com/en-us/library/w2kae45k(v=vs.80).aspx
Something like this (untested code)
[Microsoft.SqlServer.Server.SqlFunction]
public static SqlString MyFunctionName(int id) {
// Put your code here (maybe find the object you want to serialize using the id passed?)
using (var cn = new SqlConnection("context connection=true") ) {
//get your data into an object
var myObject = new {Name = "My Name"};
return new SqlString(Newtonsoft.Json.JsonConvert.SerializeObject(myObject));
}
}
I’m currently working on a project where we need to archive and trace all the modified data’s.
When a modification surrender, we have to kept theses information
Who has modified the data?
When?
And … that’s why I’m asking this question: Keep the previous
and the new value of the data.
Quickly, I have to trace every modification for every data.
Example :
I have a name field why the value “Morgan”.
When I modify this value, I have to be able to say to the user that the 6th of January, by XXX, the value changed from “Morgan” to “Robert” …
I have to find a clean and generic method to do this because a large amount of data is concerned by this behavior.
My program is in C# (.NET 4) and we are using Sql Server 2008 R2 and NHibernate for the object mapping.
Do you any ideas, experience or solution about how to do a thing like that?
I am a little confused about at what point you want to have the old vs new data available. But, this can be done within a database trigger as in the following question:
trigger-insert-old-values-values-that-was-updated
NHibernate Envers its what you want :)
You must use NHibernate 3.2+ (3.2 is the current release).
Its easy like
enversConf.Audit<Person>();
You can get info here and here
I've been in the same situation as you. I ended up doing in this way:
Save an ActivityEntry in the database containing an identity column (if you have multiple objects that change), an action-indicator (could be "User changed firstname", as a int), date field, userId and most important a parameter field.
Combining the values from the parameter field and the action-indicator I'm able to make strings like "{0} changed {1}'s firstname from {2} to {3}" where my parameter values could be "John;Joe".
I know it feels kinda wrong saving these totally loosely typed values in the database, but I believe it's the only way around, without having a copy of each table.
I stucked at a condition , where i need to share values between the pages. I want to share value from Codebehind via little or no javascript. I already have a question here on SO , but using JS. Still did'nt got any result so another approach i am asking.
So I want to know can i pass any .net object in query string. SO that i can unbox it on other end conveniently.
Update
Or is there any JavaScript approach, by passing it to windows modal dialog. or something like that.
What I am doing
What i was doing is that on my parent page load. I am extracting the properties from my class that has values fetched from db. and put it in a Session["mySession"]. Some thing like this.
Session["mySession"] = myClass.myStatus which is List<int>;
Now on one my event that checkbox click event from client side, i am opening a popup. and on its page load, extracting the list and filling the checkbox list on the child page.
Now from here user can modify its selection and close this page. Close is done via a button called save , on which i am iterating through the checked items and again sending it in Session["mySession"].
But the problem is here , when ever i again click on radio button to view the updated values , it displays the previous one. That is , If my total count of list is 3 from the db, and after modification it is 1. After reopening it still displays 3 instead of 1.
Yes, you could but you would have to serialize that value so that it could be encoded as a string. I think a much better approach would be to put the object in session rather than on the URL.
I would so something like this.
var stringNumbers = intNumbers.Select(i => i.ToString()).ToArray();
var qsValue = string.Join(",", stringNumbers);
Request.Redirect("Page.aspx?numbers=" + sqValue);
Keep in mind that if there are too many numbers the query string is not the best option. Also remember that anyone can see the query string so if this data needs to be secure do not use the query string. Keep in mind the suggestions of other posters.
Note
If you are using .NET 4 you can simplify the above code:
var qsValue = string.Join(",", intNumbers);
Make the object serializable and store it in an out-of-process session.
All pages on your web application will then be able to access the object.
you could serialize it and make it printable but you shouldn't
really, you shouldn't
The specification does not dictate a minimum or maximum URL length, but implementation varies by browser and version. For example, Internet Explorer does not support URLs that have more than 2083 characters.[6][7] There is no limit on the number of parameters in a URL; only the raw (as opposed to URL encoded) character length of the URL matters. Web servers may also impose limits on the length of the query string, depending on how the URL and query string is stored. If the URL is too long, the web server fails with the 414 Request-URI Too Long HTTP status code.
I would probably use a cookie to store the object.