How to query data when columns are encrypted in SQL Server 2016 - c#

I've been trying to find solution in this problem. I encrypted my columns from my database in SQL Server 2016 . In order to read the data I already set the parameters "Column Encryption Setting=Enabled"; I know I don't have problem in my certificate because I was able to query like SELECT * FROM TABLE, but I wasn't able to query using a WHERE condition. For example
"SELECT column FROM Table WHERE column='abc'" something like that.
The Error is:
"Operand type clash: varchar is incompatible with varchar(8000)
encrypted with (encryption_type = 'DETERMINISTIC',... and so on"
I don't know why I can't retrieve the data from the encrypted column.


I assume you have encrypted your column using the Always Encrypted feature of SQL Server.
With Always Encrypted SQL Server does not know the keys for encrypting and decrypting your data. It's the job of your client application to encrypt data before sending it to SQL Server.
When you execute this query:
SELECT column FROM Table WHERE column='abc'
you are asking SQL Server to compare the non-encrypted varchar 'abc' to the encrypted values found in your encrypted column. This will fail - as you've noted yourself.
Your client application must encrypt the value you want to use in your WHERE clause before sending it off to SQL Server.
If you're using SSMS you can do this by enabling parameterization for always encrypted. You can enable this in SSMS under Query>>Query options>>Execution>>Advanced>>Enable Parameterization for Always Encrypted.
Once this is enabled you'll be able to filter your table like this:
DECLARE #MyValue VARCHAR(100) = 'abc';
SELECT [Column] FROM [Table] WHERE [Column] = #MyValue;
What happens behind the scenes is that SSMS encrypts the value of the #MyValue parameter for you before sending it off to SQL Server. It is important that you declare and initialize the parameter at once. Otherwise SSMS is unable to encrypt the value before sending the query to SQL Server.
You can read more about parameterization for always encrypted here. Feel free to comment below if the above explanation is unclear to you.


You need to use DECRYPTBYKEY
SELECT *
FROM Table WHERE convert(varchar,DecryptByKey(column))='abc'
The way you use this function depends on if you are using symmetric keys and a hash, or just a key, etc.

Related

Update/Insert rows in SQL table with encrypted columns using LinqToSQL - Always Encrypt

I am trying to get a hands-on on Always Encrypt feature of SQL Server 2016. I am using an existing application and database to do this.
I have a table [User], where i have encrypted the 'Password' column with 'Deterministic' type. I have made the connection string changes and I am able to retrieve all the rows. I have created the repository for this table.
I am trying to insert and update rows in this table with LinqToSQL using InsertOnSubmit() and SubmitChanges().
But whenever I try to insert new rows or update existing rows, I get the error:
Msg 206, Level 16, State 2, Line 7 Operand type clash: varchar is
incompatible with varchar(20) encrypted with (encryption_type =
'DETERMINISTIC', encryption_algorithm_name =
'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name =
'CEK_Auto1', column_encryption_key_database_name = 'BRO_UAT')
collation_name = 'Latin1_General_BIN2'
I have read articles where using Stored Procs and parameterization has solved the issue. But, as I mentioned earlier, this is an existing project, and I have used LinqToSql and do not want to change the code. Insertion/Update works fine if the column is not encrypted!
Am I missing some setting?Please point me towards the right direction.

Change your column to nvarchar and check.
Also now even storing the password is not the best approach, store a signature instead and validate that signature when validating the password.
Check the link for the same.
https://social.msdn.microsoft.com/Forums/en-US/77bb69f0-590e-40f5-b5e9-714bf590e008/how-to-handle-encrypted-column?forum=linqtosql

SQL encryption only returning first character

I have the following query
UPDATE mytable
SET col1 = ENCRYPTBYPASSPHRASE ('Key', col2)
FROM mytable
when I decrypt it using
SELECT CONVERT(VARCHAR(20), DECRYPTBYPASSPHRASE ('Key', col1))
FROM mytable
The result returned is only the first character, for example if the field contains "Computer" the result is only "C".

col2 is probably nvarchar not varchar. Try
SELECT CONVERT(NVARCHAR(20), DECRYPTBYPASSPHRASE ('Key', col1))
FROM mytable
In nvarchar the code points for standard ASCII letters are the same as for ASCII but padded out with a 0x00 byte.
When you cast that to varchar that it is treated as a null character that terminates the string.

After investigation I had come to many issues so I will post what I came across, so anyone can benefit from it.
If you changed to data type of the SQL column to varbinary then make sure that when you decrypt the data, you use the same old data type. That is if you had a column of varchar that contains data and then you changed it to varbinary, you must decrypt it using varchar, if you use nvarchar ,you will get garbage data.
You must encrypt and decrypt using the same way. That is if you are loading the password from a stored procedure and use it in encrypting,and the SAME EXACT password is loaded using a function for decryption, u will also get garbage data (I tested it but I did not know why is this behaviour!)may be internally there is some difference between how data is returned from SP and functions.
Hope this helps anyone out there !

Use CONVERT with data type and size of the value you are encrypting updating.
Looks like EncryptByKey does not recognize the data properly as per column schema.
Try as below
ENCRYPTBYKEY(KEY_GUID('<Key Name>'), CONVERT(varchar(20),col1))

MySql C# Insert Error. Incorrect string value: '\xE1ra'

I'm using the C# .NET Mysql Connector, and when running this query:
INSERT INTO convos (`userid`,`time`,`from`,`content`,`read`,`deleted`, `ip`, `source`, `charname`, `to`) VALUES ('3', '1347396787', 'Chára', '........', '0', '0', '0.0.0.0:0000', 'C', 'óóóíííí', 'óóóíííí');
I get the following error:
Incorrect string value: '\xE1ra' for column 'from' at row 1
I understand my encoding, everything was configured for utf8, utf8_general_ci. Database, table and columns are configured for utf8. The data is sent from the client in utf8.
If i use a 3rd party tool like, Workbench to insert the query or use the mysql command line it works fine. I don't know if there is a bug with the connector or i need to be doing something else with the values before insert?
Any idea?
Thanks
Is there any in mysql to covert to the correct type?

I believe you need to alter the column's char set:
use below code for those columns which is using UTF-8.
ALTER TABLE database.table MODIFY COLUMN col VARCHAR(255)
CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL;

Unicode string prefix with N
First see your table convos and make sure columns data types is either one of nchar, nvarchar and You must precede all Unicode strings with a prefix N when you deal with Unicode string constants in SQL Server
Tyr:
insertQuery = "INSERT INTO convos (`userid`,`time`,`from`,`content`,`read`,`deleted`, `ip`, `source`, `charname`, `to`) VALUES
(N'3', N'1347396787', N'Chára', N'........', N'0', N'0', N'0.0.0.0:0000', N'C', N'óóóíííí', N'óóóíííí')";

I figured this out, its taken a while but it seems i was setting the charset too often. The database, tables, columns are all in UTF8. When i made a connection i had "CHARSET=UTF8" in the connection string. I was also running "SET NAMES 'utf8' COLLATE 'utf8_general_ci'" everytime i made a connection. I dropped the CHARSET=UTF8 and "SET NAMES 'utf8' COLLATE 'utf8_general_ci'" and its all working now.

Update it
INSERT INTO convos (`userid`,`time`,`from`,`content`,`read`,`deleted`, `ip`, `source`, `charname`, `to`) VALUES ('3', '1347396787', 'Chara', '........', '0', '0', '0.0.0.0:0000', 'C', 'óóóíííí', 'óóóíííí');
I think for "chara"in your 3rd value gives it

For someone who has tried all the suggestions, and nothing has worked (like myself), it is worth checking what MySQL types are your fields mapped to in C#. My text fields were automatically mapped as MySqlDbType.Blob and that was causing the error. I changed the type to MySqlDbType.Text, and I don't see the error any more.
Here is my original response to a similar thread:
https://stackoverflow.com/a/16989466/2199026

config mysql like below. it will solve the unicode error when insert into mysql from java or C#
enter image description here

Encrypt in SQL server and read/decrpyt client-side

Ive written in sql :
CREATE SYMMETRIC KEY SecureSymmetricKey
WITH ALGORITHM = TRIPLE_DES
ENCRYPTION BY PASSWORD = 'StrongPassword';
DECLARE #str NVARCHAR(1000)
SET #str = 'lala';
OPEN SYMMETRIC KEY SecureSymmetricKey
DECRYPTION BY PASSWORD = 'StrongPassword';
DECLARE #encrypted_str VARBINARY(MAX)
SET #encrypted_str =
EncryptByKey(Key_GUID('SecureSymmetricKey'), #str);
the encrypted_str value is in a table now.
how can i read it in c# ? ( and decrypt in c# )

You don't "decrypt" it in C#. Decrypt it in the select statement on the way back out.

You need to encrypt and decrypt at the same layer of your application stack; if you encrypt at the SQL Server layer, you need to decrypt before returning the data to your application.
If you want to decrypt it in the C# application, you need to encrypt it there first, and store the encrypted values in the database.

Encrypt Data in one "slot" in database

I read over a great tutorial on how to encrypt the data using a symmetrical key in one column.
Click here to view it.
This has helped me set up a column for encryption, which is what I need. My problem is that I will constantly be updating rows in my database - so copying over one whole original column to the encrypted columm like the tutorial shows doesn't work. Is there a way that I can insert a value that is given to me (using C#/asp) and direclty encrypt it as I insert it into the database, rather than having to place it in one column and copying it over, and then dropping the other column?

Just call EncryptByKey on your data to encrypt You don't need another column. In this case it has to be included in your SQL which should be a parameterized query or a stored procedure.
Insert into Whatever
YourEncryptedData
Values ( EncryptByKey(...))
You may have top open your key first depending on your implementation.
http://msdn.microsoft.com/en-us/library/ms174361.aspx

You need to have the certificate open do do the encryption. This is from the page you linked to:
OPEN SYMMETRIC KEY TestTableKey
DECRYPTION BY CERTIFICATE EncryptTestCert
UPDATE TestTable
SET EncryptSecondCol = ENCRYPTBYKEY(KEY_GUID('TestTableKey'),SecondCol)
Inserting works the same:
OPEN SYMMETRIC KEY TestTableKey
DECRYPTION BY CERTIFICATE EncryptTestCert
INSERT INTO TestTable (FirstCol, EncryptSecondCol)
VALUES (6, ENCRYPTBYKEY(KEY_GUID('TestTableKey'),'Sixth'))
You can execute the open key command before doing your insert or create a stored procedure to do the insert.

Categories

Resources