My application is MVC 5 using EF 6.2. I am decrypting certain columns while generating a list, it works but slow. Is there a better way to improve the performance of this approach?
var mylist = await _db.vw_LearnerCourse.AsNoTracking().ToListAsync();
var grid1 = mylist.Select(c => new
{
FirstName = Encryption.Decrypt5(c.FirstName),
LastName = Encryption.Decrypt5(c.LastName)
}).ToList();
public static string Decrypt5(string cipherText)
{
if (string.IsNullOrWhiteSpace(cipherText)) return null;
if (!string.IsNullOrWhiteSpace(cipherText))
{
string strOut;
try
{
var arrOffsets = new ArrayList();
arrOffsets.Insert(0, 73);
arrOffsets.Insert(1, 56);
arrOffsets.Insert(2, 31);
arrOffsets.Insert(3, 58);
arrOffsets.Insert(4, 77);
arrOffsets.Insert(5, 75);
strOut = "";
int intCounter;
for (intCounter = 0;
intCounter <= cipherText.Length - 1;
intCounter += 2)
{
var strSub = cipherText.Substring(intCounter, 1);
var strSub1 = cipherText.Substring(intCounter + 1, 1);
var intVal = int.Parse(strSub,
NumberStyles.HexNumber) * 16 + int.Parse(strSub1,
NumberStyles.HexNumber);
var intMod = intCounter / 2 % arrOffsets.Count;
var intNewVal = intVal -
Convert.ToInt32(arrOffsets[intMod]) + 256;
intNewVal = intNewVal % 256;
var strDecimal = ((char)intNewVal).ToString();
strOut = strOut + strDecimal;
}
}
catch (Exception err)
{
throw new Exception(err.Message);
}
var encryptionKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
cipherText = strOut;
cipherText = cipherText.Replace(" ", "+");
var cipherBytes = Convert.FromBase64String(cipherText);
using (var encryptor = Aes.Create())
{
var pdb = new Rfc2898DeriveBytes(encryptionKey, new byte[]
{
xxxxxxx
});
encryptor.Key = pdb.GetBytes(32);
encryptor.IV = pdb.GetBytes(16);
using (var ms = new MemoryStream())
{
using (var cs = new CryptoStream(ms, encryptor.CreateDecryptor(),
CryptoStreamMode.Write))
{
cs.Write(cipherBytes, 0, cipherBytes.Length);
cs.Close();
}
cipherText = Encoding.Unicode.GetString(ms.ToArray());
}
}
return cipherText;
}
return null;
}
The main reason for the poor performance is the execution of the PBKDF2 key derivation (via Rfc2898DeriveBytes) for every encryption. Key derivations are intentionally slow to slow down attackers. I.e. each encryption adds an artificial delay, which leads to the observed performance loss over runtime.
The execution time of PBKDF2 can be tuned via the iteration count. In general, the value is set as high as possible while maintaining acceptable performance. A typical value is 10,000 iterations, see e.g. here.
The posted code uses the default value of 1000 iterations (see here), which is already very low. Reducing the value also reduces security and is therefore less recommended.
Apart from that, changing the iteration count would lead to incompatibility with already existing data, so a data migration would be necessary.
Another alternative to improve performance is to perform key derivation on list level (instead of for each encryption of the list) and pass key and IV to the Decrypt5() method. This also ensures that compatibility with the old data is not lost:
// Key/IV derivation
var encryptionKey = "This is my passphrase";
var salt = new byte[] {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07};
var pdb = new Rfc2898DeriveBytes(encryptionKey, salt);
byte[] key = pdb.GetBytes(32);
byte[] iv = pdb.GetBytes(16);
// Decryption, key/IV are passed
var mylist = await _db.vw_LearnerCourse.AsNoTracking().ToListAsync();
var grid1 = mylist.Select(c => new
{
FirstName = Encryption.Decrypt5(c.FirstName, key, iv),
LastName = Encryption.Decrypt5(c.LastName, key, iv)
}).ToList();
...
// Decryption without key/IV derivation
public static string Decrypt5(string cipherText, byte[] key, byte[] iv)
{
...
var cipherBytes = Convert.FromBase64String(cipherText);
using (var encryptor = Aes.Create())
{
encryptor.Key = key;
encryptor.IV = iv;
using (var ms = new MemoryStream())
{
using (var cs = new CryptoStream(ms, encryptor.CreateDecryptor(), CryptoStreamMode.Write))
{
cs.Write(cipherBytes, 0, cipherBytes.Length);
cs.Close();
}
cipherText = Encoding.Unicode.GetString(ms.ToArray());
}
}
return cipherText;
...
}
This change significantly improves performance while maintaining compatibility with existing data.
Security:
A vulnerability of the current code is the static salt. A static salt is generally insecure because in case of a successful attack the entire data is affected. Since in the current case the key derivation additionally derives the IV besides the key, this is further aggravated by the inevitable key/IV reuse.
To increase security, these vulnerabilities should be eliminated. However, it is impossible to avoid losing compatibility with the old data, so data migration is necessary.
To improve security, instead of the static salt, a random salt should be generated for each encryption or here with regard to performance at least for each list.
To avoid reuse of key/IV pairs within a list, the IV should not be derived via key derivation, but a fresh, random IV should be generated for each encryption. Although the generation of the random IV has a negative impact on performance, this is significantly outweighed by the gain due to the shift of the key derivation.
Salt and IV are not secret and are usually concatenated with the ciphertext: salt|IV|ciphertext. During decryption, the portions are separated based on the known lengths of salt and IV. Since the same salt is used here for all encryptions of a list, the salt could also be stored separately.
For the iteration count not the default value (of 1000 iterations) should be used, but the largest possible value with acceptable performance as well.
Hard coding of the password should be refrained from (otherwise, e.g. access to the code is enough to compromise the password), instead the password should be entered at runtime.
You may create a Decryptor instance every time when you Decrypt a string. Try create a static decryptor, and use this instance every time.
TripleDESCryptoServiceProvider des = new TripleDESCryptoServiceProvider
{
Key = key,
Mode = CipherMode.ECB
};
// use this instance all time
var decryptor = des.CreateDecryptor();
var mylist = await _db.vw_LearnerCourse.AsNoTracking().ToListAsync();
var grid1 = mylist.Select(c => new
{
FirstName = Encryption.Decrypt5(decryptor,c.FirstName),
LastName = Encryption.Decrypt5(decryptor,c.LastName)
}).ToList();
public static string Decrypt5(ICryptoTransform decryptor, string cipherText)
{
if (string.IsNullOrWhiteSpace(cipherText)) return null;
if (!string.IsNullOrWhiteSpace(cipherText))
{
xxxxxxxx
}
The one thing I'm certain there is no need for manual Encryption\Decryption.
OPTION1
You should use EncryptedColumns EF feature and let encryption/decryption be handled naturally.
https://sd.blackball.lv/articles/read/18805 see example of implementation. Github project https://github.com/emrekizildas/EntityFrameworkCore.EncryptColumn.
Init context with encryption key:
private readonly IEncryptionProvider _provider;
public ExampleDbContext()
{
Initialize.EncryptionKey = "example_encrypt_key";
this._provider = new GenerateEncryptionProvider();
}
Let modelBinder know about encryption:
modelBuilder.UseEncryption(this._provider);
Set the columns
public class User
{
public Guid ID { get; set; }
public string Firstname { get; set; }
public string Lastname { get; set; }
[EncryptColumn]
public string EmailAddress { get; set; }
[EncryptColumn]
public string IdentityNumber { get; set; }
}
OPTION2
You can also opt for Always Encrypted, this will encrypt entire DB
https://techcommunity.microsoft.com/t5/sql-server-blog/using-always-encrypted-with-entity-framework-6/ba-p/384433 for details and some limitations of approach.
The idea is straightforward, we delegate all encryption process to database (for ex MS SQL Server) if it supports this and using SQL Server features organize encryption. The EF Core we instruct via connection string that DB is encrypted and that's really it. Noticeable difference to normal DB design:
Entity Framework assumes order-comparability of PKs in many cases. If PK is encrypted, some scenarios will not work.
Also EF will sometimes print values of the Entity key in exception messages. This could cause sensitive information to be leaked to inappropriate parties. (issue tracked, MS recommended to use surrogate keys)
EF query will fail if we compare encrypted column to a constant, instead need to pass constant argument as closure
EF can transparently replace constants in the query with parameters. This can be achieved using query interception extensibility features.
Sorting based on encrypted column is not supported on the database due to limitations of Always Encrypted feature. This need to be done on client side.
Some GroupBy operations are not supported (namely the LINQ specific grouping, without projecting group key or aggregate function) if entity key is encrypted. Reason is that those queries (that simply aggregate elements into IGrouping<,> statements) are translated into TSQL containing ORDER BY operation on the key. If the key is encrypted, the query will fail. You have to use unencrypted surrogate key, to let OrderBy work
Queries that project a collection don’t work with encrypted columns if the key (or any part of the composite key) is encrypted. The reason is same OrderBy not work for encrypted columns. Solution as previous omit encrypting columns that need server side sorting.
Similarly to the above case, Include operation performed on a collection is not supported if the PK of the principal entity is encrypted
Perhaps try and use a concrete class instead of a dynamic object. Dynamic objects create some overhead and if there's a lot of them I guess that could be what slows it down
Like so:
class Names
{
public string? FirstName { get; }
public string? LastName { get; }
public Names(string? firstName, string? lastName)
{
FirstName = firstName;
LastName = lastName;
}
}
var grid1 = mylist.Select(c =>
new Names(
Encryption.Decrypt5(c.FirstName),
Encryption.Decrypt5(c.LastName))
.ToList();
It also looks like you're unnecessarily checking if cipherText is null or white space twice
For an application purpose, I want to store an complete content of the email (string) in the dictionary.
[I know that this is what every hash function provides but wanted to explicitly state that the hash for the same string should always be the same]
Since its not for cryptographic reason and only for storing in dictionary. Can any one please suggest a good hashing function that is available in .Net. My concern is that the email string can be pretty big and i want my hash function to support the big string and not cause frequent collision. I am looking for storing around 500 entries.
Please note i dont want to write my own hash funciton but leverage an existing availaible hash function in .Net
You may consider to use HashAlgorithm.ComputeHash.
Here is an example which is provided with this function:
using System;
using System.Security.Cryptography;
using System.Text;
public class Program
{
public static void Main()
{
string source = "Hello World!";
using (SHA256 sha256Hash = SHA256.Create())
{
string hash = GetHash(sha256Hash, source);
Console.WriteLine($"The SHA256 hash of {source} is: {hash}.");
Console.WriteLine("Verifying the hash...");
if (VerifyHash(sha256Hash, source, hash))
{
Console.WriteLine("The hashes are the same.");
}
else
{
Console.WriteLine("The hashes are not same.");
}
}
}
private static string GetHash(HashAlgorithm hashAlgorithm, string input)
{
// Convert the input string to a byte array and compute the hash.
byte[] data = hashAlgorithm.ComputeHash(Encoding.UTF8.GetBytes(input));
// Create a new Stringbuilder to collect the bytes
// and create a string.
var sBuilder = new StringBuilder();
// Loop through each byte of the hashed data
// and format each one as a hexadecimal string.
for (int i = 0; i < data.Length; i++)
{
sBuilder.Append(data[i].ToString("x2"));
}
// Return the hexadecimal string.
return sBuilder.ToString();
}
// Verify a hash against a string.
private static bool VerifyHash(HashAlgorithm hashAlgorithm, string input, string hash)
{
// Hash the input.
var hashOfInput = GetHash(hashAlgorithm, input);
// Create a StringComparer an compare the hashes.
StringComparer comparer = StringComparer.OrdinalIgnoreCase;
return comparer.Compare(hashOfInput, hash) == 0;
}
}
I hope it helps 😊
My exercise is to port an administration-backend from php to .net.
The backend communicates with an app written in java.
Some things compared with md5-hashes, in php and java the md5 hashes are equally.
I cannot change the md5 hash-code in the java app, because then will over 10k customer cards not work.
My problem is, the backend is ported and now the communication between the new backend (.net) and the java app.
My .net md5-hash code returns not the same hash as the java code.
java:
public static String getMD5(String input) {
try {
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] messageDigest = md.digest(input.getBytes());
BigInteger number = new BigInteger(1, messageDigest);
String hashtext = number.toString(16);
// Now we need to zero pad it if you actually want the full 32 chars.
while (hashtext.length() < 32)
hashtext = "0" + hashtext;
return hashtext;
}
catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
}
and my .net code:
public String hashMD5(String wert)
{
byte[] asciiBytes = ASCIIEncoding.UTF8.GetBytes(wert);
byte[] hashedBytes = MD5CryptoServiceProvider.Create().ComputeHash(asciiBytes);
string hashedString = BitConverter.ToString(hashedBytes).Replace("-", "").ToLower();
return hashedString;
}
my java code returns for bb27aee4 :
46d5acfcd281bca9f1df7c9e38d50576
and my .net code returns:
b767fe33172ec6cbea569810ee6cfc05
I don't know what I have to do...
Please help and thanks in advance.
Its not the problem of MD5 hash generator
MD5 hash for bb27aee4 : 46d5acfcd281bca9f1df7c9e38d50576 &
MD5 hash for BB27AEE4 : b767fe33172ec6cbea569810ee6cfc05
So basically in .NET you are generating MD5 hash for BB27AEE4 instead of bb27aee4
So check for the error in your code
Good Luck
The reason you're seeing different results from your hash is because the encoding used for your string differs. In your .NET code, you specify UTF8 explicitly, while there is no guarantee your Java code does the same; it may very well be using ASCII, which results in different hashes being generated.
If you explicitly specify the same encoding in both your .NET and Java code, you'll end up with the same MD5 hashes as well.
I am computing md5hash of files to check if identical so I wrote the following
private static byte[] GetMD5(string p)
{
FileStream fs = new FileStream(p, FileMode.Open);
HashAlgorithm alg = new HMACMD5();
byte[] hashValue = alg.ComputeHash(fs);
fs.Close();
return hashValue;
}
and to test if for the beginning I called it like
var v1 = GetMD5("C:\\test.mp4");
var v2 = GetMD5("C:\\test.mp4");
and from debugger I listed v1 and v2 values and they are different !! why is that ?
It's because you're using HMACMD5, a keyed hashing algorithm, which combines a key with the input to produce a hash value. When you create an HMACMD5 via it's default constructor, it will use a random key each time, therefore the hashes will always be different.
You need to use MD5:
private static byte[] GetMD5(string p)
{
using(var fs = new FileStream(p, FileMode.Open))
{
using(var alg = new MD5CryptoServiceProvider())
{
return alg.ComputeHash(fs);
}
}
}
I've changed the code to use usings as well.
From the HMACMD5 constructor doc:
HMACMD5 is a type of keyed hash algorithm that is constructed from the
MD5 hash function and used as a Hash-based Message Authentication Code
(HMAC). The HMAC process mixes a secret key with the message data,
hashes the result with the hash function, mixes that hash value with
the secret key again, and then applies the hash function a second
time. The output hash will be 128 bits in length.
With this constructor, a 64-byte, randomly generated key is used.
(Emphasis mine)
With every call to GetMD5(), you're generating a new random key.
You might want to use System.Security.Cryptography.MD5Cng
My guess is that you did something like:
Console.WriteLine(v1);
Console.WriteLine(v2);
or
Console.WriteLine(v1 == v2);
That just shows that the variable values refer to distinct arrays - it doesn't say anything about the values within those arrays.
Instead, try this (to print out the hex):
Console.WriteLine(BitConverter.ToString(v1));
Console.WriteLine(BitConverter.ToString(v2))
Use ToString() methode to get the value of the array byte
I've tried every example I can find on the web but I cannot get my .NET code to produce the same MD5 Hash results from my VB6 app.
The VB6 app produces identical results to this site:
http://www.functions-online.com/md5.html
But I cannot get the same results for the same input in C# (using either the MD5.ComputeHash method or the FormsAuthentication encryption method)
Please help!!!!
As requested, here is some code. This is pulled straight from MSDN:
public string hashString(string input)
{
// Create a new instance of the MD5CryptoServiceProvider object.
MD5 md5Hasher = MD5.Create();
// Convert the input string to a byte array and compute the hash.
byte[] data = md5Hasher.ComputeHash(Encoding.Default.GetBytes(input));
// Create a new Stringbuilder to collect the bytes
// and create a string.
StringBuilder sBuilder = new StringBuilder();
// Loop through each byte of the hashed data
// and format each one as a hexadecimal string.
for (int i = 0; i < data.Length; i++)
{
sBuilder.Append(data[i].ToString("x2"));
}
// Return the hexadecimal string.
return sBuilder.ToString();
}
My test string is:
QWERTY123TEST
The results from this code is:
8c31a947080131edeaf847eb7c6fcad5
The result from Test MD5 is:
f6ef5dc04609664c2875895d7da34eb9
Note: The result from the TestMD5 is what I am expecting
Note: I've been really, really stupid, sorry - just realised I had the wrong input. As soon as I hard-coded it, it worked. Thanks for the help
This is a C# MD5 method that i know works, i have used it to authenticate via different web restful APIs
public static string GetMD5Hash(string input)
{
System.Security.Cryptography.MD5CryptoServiceProvider x = new System.Security.Cryptography.MD5CryptoServiceProvider();
byte[] bs = System.Text.Encoding.UTF8.GetBytes(input);
bs = x.ComputeHash(bs);
System.Text.StringBuilder s = new System.Text.StringBuilder();
foreach (byte b in bs)
{
s.Append(b.ToString("x2").ToLower());
}
return s.ToString();
}
What makes the "functions-online" site (http://www.functions-online.com/md5.html) an authority on MD5? For me, it works OK only for ISO-8859-1. But when I try pasting anything other than ISO-8859-1 into it, it returns the same MD5 hash. Try Cyrillic capital B by itself, code point 0x412. Or try Han Chinese symbol for water, code point 0x98A8.
As far as I know, the posted C# applet is correct.