I have a long set of SQL scripts. They are all update statements. It's for an access database. I want to validate the script before I run it. Firstly, I'd like to make sure that the query can be parsed. I.e. that the SQL is at least syntactically correct. Secondly, I'd like to make sure that the query is valid in terms of database structure - i.e. there are no missing columns or the columns are of the wrong type etc. But, I don't want the query to be actually executed. The aim of this is to do a quick validation before the process kicks off because the process takes several hours and one syntactical error can waste a day of someone's time.
I will probably write the tool in C# with .net but if there's a pre-built tool that would be even better. I will probably use the Access API. In SQL Server this is very straight forward. You can just validate the query in SQL Server management studio before running it. It will give you a good indication of whether the SQL will complete or not.
How would I go about doing this?
Edit: an answer below solves the issue of checking syntax. However, I'd still like to be able to validate the semantic content of the query is OK. However, I think this might be impossible in Access without actually running the query. Please tell me I'm wrong.
I'm not 100% sure if Access works the same way as a traditional database, but with a mainstream RDMBS, there are actually three distinct steps that happen when you run a query:
Prepare
Execute
Fetch
Most are oblivious to the distinction because they just hit "run" and see results come back.
It's the "Execute" that actually compiles the statement before going off and pulling data.
When you use ADO, you can actually see the three events as three separate calls to the database. What this means is you can trap the execute step to see if it fails, and if it succeeds, there is nothing requiring you to actually get the results.
OleDbConnection conn = new OleDbConnection();
conn.ConnectionString = String.Format("{0}{1}",
#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=", #"c:\Access\MyDb.accdb");
conn.Open();
bool valid;
using (OleDbCommand cmd = new OleDbCommand("select [Bad Field] from [Table]", conn))
{
try
{
OleDbDataReader reader = cmd.ExecuteReader();
valid = true;
reader.Close(); // Did not ever call reader.Read()
}
catch (Exception ex)
{
valid = false;
}
}
And now valid indicates whether or not the statement compiled.
If you want to get really fancy, you can parse the exception results to find out why the command failed.
Access supports transactions on its Connection object. Try to execute your SQL statement inside a transaction and always call Rollback. Wrap the whole attempt in a Try/Catch block to assess whether the statement executed successfully or not.
Related
I have a problem in my program that's supposed to store projects given by the user in a database. I'm stuck on the edit project button. After entering new values in the program and hitting the button to save the values everything runs successfully with no errors. The messagebox that says "project edited" appears, I hit ok but the database stays the same. There is no error in the code, the SQL code that gets sent to update the database values is also correct but it doesn't work. Can anyone help with this because I am lost.
Here is the method that creates and executes the SQL code to update the database.
enter image description here
Wow man that code is wrong in so many ways according to code standards and principles most popular :) but that is not what the question is about directly, though getting you past lost we have to start at the basic tbh:
Suggestions
when you catch that exception if it comes, show that in a messagebox also you can even add an error icon as part of the .Show command, it's build in.
Move the connection.Close to the finally block instead of having it replicated
Consider making an SQL procedure instead and just parse the parameter into that, this code is prone to sql injection that you pose
Consider not making the procedure and familiarize Yourself with entity framework, it's going to make your life so much easier
do not concatenate like that, use interpolation or string.Combine or you'll be copying stuff all around on the stack, for each + a new copy one and two into third, it is super inefficient
When You write the code works and the sql is correct, consider that the outcome is not the desired and therefore it doesn't technically ;) the best and the worst about computers, is that they do what you ask.
Don't write Your DAL code in the form at all
Consider checking your parameters for default values
You do not have data to say 'project was updated' only 'values were saved', you do not check the old values in the code
Still besides that I do not see why what you wrote wouldn't work, provided the resulting sql is valid in what db you use, but i suppose if you do some of these things the error will present itself
I don't think it's a connection problem because I have a function that updates only the finishdate of the project and that works completely fine.
Here is the function:
public static MySqlCommand FinishProject(int projID, string finishdate) {
try {
if (connection != null) {
connection.Open();
cmd = connection.CreateCommand();
cmd.CommandType = CommandType.Text;
cmd.Parameters.AddWithValue("#value", projID);
cmd.Parameters.AddWithValue("#finishdate", finishdate);
cmd.CommandText = "UPDATE `b1c`.`projects` SET `finishdate` = (#finishdate) WHERE (`projectid` = (#value));";
int i = cmd.ExecuteNonQuery();
connection.Close();
if (i != 0) {
MessageBox.Show("Project finalized.");
i = 0;
}
}
} catch (Exception ex) {
MessageBox.Show("Catch");
connection.Close();
}
return cmd;
}
You can see it's basically the same the only difference are the values.
So it shouldn't be a connection thing because this one works fine I think.
I also don't think it's a problem in the SQL database because all the problems
I've had up until now that had anything to do with the database have shown as errors in visual studio.
If anyone can help I will provide screenshots of anything you need and thank you all once again for trying to help.
Here is the screenshot of the function I've pasted previously so it's easier to look at.
finishprojectfunction
This is going to top some of the weirdest things I've seen. I've tried looking up "simple t-sql delete causing timeout" but all titles are misleading, they say simple but are not. They deal with deleting millions of records or have complex relationships setup. I do not.
I have four tables:
tblInterchangeControl,
tblFunctionalGroup,
tblTransactionSet,
tblSegment
The latter 3 all associate to tblInterchangeConrol via InterchangeControlID. There are no relationships setup. like I said as simple as one could get.
The procedure runs a delete statement on all 4 tables like so...
DELETE FROM tblSegment
WHERE (ID_InterchangeControlID = #InterchangeControlID)
DELETE FROM tblTransactionSet
WHERE (ID_InterchangeControlID = #InterchangeControlID)
DELETE FROM tblFunctionalGroup
WHERE (ID_InterchangeControlID = #InterchangeControlID)
DELETE FROM tblInterchangeControl
WHERE (InterchangeControlID = #InterchangeControlID)
The weird part is if I leave these in the procedure it times out, if I remove them, it does not. I've pinned it to these delete statements that are the cause. But Why?!
I included c# because I'm calling this procedure from a c# application. I don't think this is the issue but maybe. I only say I don't think so because my code work just fine when I remove the delete statements inside the stored procedure. Then if I put them back, an exception is thrown that it's timed out.
In case my comment is the answer.
Most likely you have some locks holding those deletes up.
If you run a query from a command line SQL tool or from SQL Management Studio it will take whatever it needs to complete the query. So yes, most likely it's client part issue. And, because you mentioned c# it's probably ADO.NET command timeout.
Also, I'd suggest to profile the queries by inspecting their execution plans. In case you don't have indexes (primary/unique key constraints) this will result to full-scan, i.e. O(n) operation you don't want.
Update:
OK, looks like it's ADO.NET error. In your code, just prior executing the command increase the timeout:
var myCommand = new SqlCommand("EXEC ..."); // you create it something like this
....
myCommand.CommandTimeout = 300; // 5 minutes
myCommand.ExecuteNonReader(); // assuming your SP doesn't return anything
I've inherited an application with a lot of ADO work in it, but the insert/update helper method that was written returns void. We've also been experiencing a lot of issues with data updates/inserts not actually happening. My goal is to update all of them to check rows affected and depending on the results, act accordingly, but for the time being of finding what may be causing the issue, I wanted to log SQL statements that are called against the server and the number of rows affected by the statement.
This is the statement I'm attempting:
SqlCommand com = new SqlCommand(String.Format("'INSERT INTO
SqlUpdateInsertHistory(Statement, AffectedRows) VALUES (''{0}'', {1});'",
statement.Replace("'", "''"), rows), con);
but it seems to constantly break somewhere in the sql that is being passed in (some cases on single quotes, but I imagine there are other characters that could cause it as well.
Is there a safe way to prep a statement string to be inserted?
I just can't rightly propose a solution to this question without totally modifying what you're doing. You're currently wide open to SQL Injection. Even if this is a local application, practice how you want to play.
using (SqlCommand com = new SqlCommand("INSERT INTO SqlUpdateInsertHistory(Statement, AffectedRows) VALUES (#Statement, #AffectedRows)", con))
{
com.Parameters.AddWithValue("#Statement", statement);
com.Parameters.AddWithValue("#AffectedRows", rows);
com.ExecuteNonQuery();
}
Have you tried SQL Server Profiler? It's already been written and logs queries, etc.
Someone else tried this and got a lot of decent answers here.
I know how to execute queries from C# but I want to provide a dropdown list in which people can write a query and it will execute and populate the list.
A problem is that I want to forbid all queries that modify the database in any way. I have not managed to find a way to do this and I did my best with google.
The solution I can think of is that I will scan the query for INSERT, DELETE, UPDATE and only allow SELECT statements. However, I want to be able to allow users to call stored procedures as well. This means I need to get the body of the stored procedure and scan it before I execute it. How do I download a stored procedure then?
If anyone knows a way to only execute read only queries do share please! I have the feeling scanning the text for INSERT, DELETE, UPDATE doesn't prevent SQL injections.
The easiest way to do this might be to offload this job to the database. Just make sure that the database user that will be running the queries has read-access only. Then, any queries that do anything other than SELECT will fail, and you can report that failure back to the users.
If you don't go this route, the complexity becomes quite enormous, since you basically have to be prepared to parse an arbitrary SQL statement, not to mention arbitrary sequences of SQL statements if you allow stored procs to be run.
Even then, take care to ensure that you aren't leaking sensitive data through your queries. Directly input queries from site users can be dangerous if you're not careful. Even if you are, allowing these queries on anything but a specifically constructed sandbox database is a "whoops, I accidentally changed the user's permissions" away from becoming a security nightmare.
Another option is to write a "query creator" page, where users can pick the table and columns they'd like to see. You can then a) only show tables and columns that are appropriate for a given user (possibly based on user roles etc.) and b) generate the SQL yourself, preferably using a parameterized query.
Update: As Yahia points out, if the user has execute privilege (so that they can execute stored procs,) then the permissions of the procedure itself are honoured. Given that, it might be better to not allow arbitrary stored proc execution, but rather offer the users a list of procedures that are known to be safe. That will probably be difficult to maintain and error-prone, though, so disallowing stored procs altogether might be best.
How about creating a user account on the database server which only has select (read-only) rights?
Perhaps you could set up a SQL user with read-only access to the database and issue the command using that user? Then you can catch the errors when/if they happen.
It seems to me that it's going to be very difficult and error-prone to try to parse the query to figure out if it modifies the database.
You can't parse SQL like that reliably.
Use permissions to
Allow only SELECT on tables and views
No permissions on stored procedures that change data (An end user by default won't be able to see stored procedure definition)
Best is to not allow users to enter SQL and use only prepared/parameterized queries...
The next best way to prevent that is to use a restricted user with pure read access
The above two can be combined...
BEWARE
To execute a Stored Procedure the user must have execute privilege... IF the Stored Procedure modifies data then this would happen without an error messages even with a restricted user since the permission to modify is granted to the Stored Procedure!
IF you absolutely must allow users to enter SQL and can't restrict the login then you would need to use a SQL parser - for example this...
As to how to download the body of a Stored Procedure - this is dependent on the DB you use (SQL Server, Oracle etc.).
EDIT:
Another option are so-called "Database Firewall" - you connect instead of directly to the DB to the Firewall... in the Firewall you configure several things like time-based restrictions (when specific users/statement are/art not allowed), SQL-based statement (which are allowed...), quantity-based restrictions (like you can get 100 records, but are not able to download the whole table/DB...) etc.
There are commercial and opensource DB Firewalls out there - though these are by nature very dependent on the DB you use etc.
Examples:
Oracle Firewall - works with Oracle / SQL Server / DB2 etc.
SecureSphere - several including Oracle / SQL Server / DB2 etc.
GreenSQL - opensource version support Postgres + MySQL, commercial MS SQL Server
Don't forget about things that are even worse than INSERT, UPDATE, and DELETE. Like TRUNCATE...that's some bad stuff.
i think SQL Trigger is the best way what you want to do.
Your first move should be to create a DB user for this specific task with only the needed permissions (basically SELECT only), and with the rights to see only the tables you need them to see (so they cannot SELECT sys tables or your users table).
More generally, it seems like a bad idea to let users execute code directly on your database. Even if you protect it against data modification, they will still be able to make ugly-looking joins to make your db run slow, for instance.
Maybe whichever language your programming the UI with, you could try to look online for a custom control that allows filtering on a database. Google it...
this is not perfect but might be what you want, this allows the keyword to appear if its a part of a bigger alphanumeric string:
public static bool ValidateQuery(string query)
{
return !ValidateRegex("delete", query) && !ValidateRegex("exec", query) && !ValidateRegex("insert", query) && !ValidateRegex("alter", query) &&
!ValidateRegex("create", query) && !ValidateRegex("drop", query) && !ValidateRegex("truncate", query);
}
public static bool ValidateRegex(string term, string query)
{
// this regex finds all keywords {0} that are not leading or trailing by alphanumeric
return new Regex(string.Format("([^0-9a-z]{0}[^0-9a-z])|(^{0}[^0-9a-z])", term), RegexOptions.IgnoreCase).IsMatch(query);
}
you can see how it works here: regexstorm
see regex cheat sheet: cheatsheet1, cheatsheet2
notice this is not perfect since it might block a query with one of the keywords as a quote, but if you write the queries and its just a precaution then this might do the trick.
you can also take a different approach, try the query, and if it affects the database do a rollback:
public static bool IsDbAffected(string query, string conn, List<SqlParameter> parameters = null)
{
var response = false;
using (var sqlConnection = new SqlConnection(conn))
{
sqlConnection.Open();
using (var transaction = sqlConnection.BeginTransaction("Test Transaction"))
using (var command = new SqlCommand(query, sqlConnection, transaction))
{
command.Connection = sqlConnection;
command.CommandType = CommandType.Text;
command.CommandText = query;
if (parameters != null)
command.Parameters.AddRange(parameters.ToArray());
// ExecuteNonQuery() does not return data at all: only the number of rows affected by an insert, update, or delete.
if (command.ExecuteNonQuery() > 0)
{
transaction.Rollback("Test Transaction");
response = true;
}
transaction.Dispose();
command.Dispose();
}
}
return response;
}
you can also combine the two.
I have a reports page where you can enter the query manually for a report. How can I block any INSERT, UPDATE or DELETE statements, and only run SELECT?
using (var connection = new SQLiteConnection(connectionString))
{
var da = new SQLiteDataAdapter
{
SelectCommand = new SQLiteCommand(query, connection)
};
try
{
da.Fill(table);
}
I could check if the query string contains "INSERT", "UPDATE" or "DELETE", but I don't think it's a good practice.
You could use an EXPLAIN statement to break the query down into VM instructions and examine the opcode column of the output. If the value "OpenWrite" occurs then the query is not read-only.
Checking the query string is not good practice? Compared to what? Allowing a user to enter any SQL statement they want to in your report page? I can't think of a much worse practice than that. If you're going to allow that sort of thing, you absolutely need to somehow restrict the types of statements they enter, and maybe require a Where clause (to avoid millions of rows being returned) etc.
in fact did you check what happens when you try to fill the table with the data adapter having anything else than a select in the query variable? I doubt you get an empty table or dataset, I would expect an exception in which case you could rollback the transaction.
I would anyway try to create the connection as readonly as suggested above by Sorax and I would actually parse the query variable as well.
Since the SQlite database is just one file, my guess is that you can make the database readonly through the filesystem. This is of course not a fancy solution but is one that does not require any code (of course except when you're throwing exceptions when writing isn't possible).
A) Use a read-only connection - I think that would be almost the best solution
B) Use more than one TextBox as Input (but this would become more a solution like checking the String for Insert etc.)
For Example
Select |____| From |_________| Where |_______|
Edit: to answer your comment just have a look at http://www.sqlite.org/c3ref/open.html especially the topic "SQLITE_OPEN_READONLY" - I haven't done anything with sqlite now, but I think that should do the trick...