Im trying to encrypt a large file (Camtasia.exe) with the AES encryption.
Now for some reason I get a "Out of Memory" Exception. Im really new to this and I don't know how I could possibly fix that. This is my code
I use this to call my encryption method.
bytes = File.ReadAllBytes("Camtasia.exe");
Cryptography.Encryption.EncryptAES(System.Text.Encoding.Default.GetString(bytes), encryptionKey);
This is the AES encryption itself
public static string EncryptAES(string content, string password)
{
byte[] bytes = Encoding.UTF8.GetBytes(content);
using (SymmetricAlgorithm crypt = Aes.Create())
using (HashAlgorithm hash = MD5.Create())
using (MemoryStream memoryStream = new MemoryStream())
{
crypt.Key = hash.ComputeHash(Encoding.UTF8.GetBytes(password));
// This is really only needed before you call CreateEncryptor the second time,
// since it starts out random. But it's here just to show it exists.
crypt.GenerateIV();
using (CryptoStream cryptoStream = new CryptoStream(
memoryStream, crypt.CreateEncryptor(), CryptoStreamMode.Write))
{
cryptoStream.Write(bytes, 0, bytes.Length);
}
string base64IV = Convert.ToBase64String(crypt.IV);
string base64Ciphertext = Convert.ToBase64String(memoryStream.ToArray());
return base64IV + "!" + base64Ciphertext;
}
}
Here is the error again that I get when calling the function "EncryptAES" at the top. I would be glad if someone could explain how this happens and how to solve it
https://imgur.com/xqcLsKW
You're reading the entire exe into memory, interpreting it as a UTF-16 string (??!), turning that back into UTF-8 bytes, and encrypting those. This converting to/from a string is horifically broken. An executable file is not a human-readable string, and even if it was, you're in a real muddle as to which encoding you're using. I think you can drop the whole string thing.
You're also reading the entire thing into memory (several times in fact, because of the whole string thing), which is wasteful. You don't need to do that: you can encrypt it bit-by-bit. To do this, use a Stream.
Something like this should work (untested): at least it gets the general concept across. We set up a series of streams which lets us read the data out of the input file bit-by-bit, and write them out to the output file bit-by-bit.
// The file we're reading from
using var inputStream = File.OpenRead("Camtasia.exe");
// The file we're writing to
using var outputStream = File.OpenWrite("EncryptedFile.txt");
using var HashAlgorithm hash = MD5.Create();
using var aes = Aes.Create();
aes.Key = hash.ComputeHash(Encoding.UTF8.GetBytes(password));
// Turn the IV into a base64 string, add "!", encode as UTF-8, and write to the file
string base64IV = Convert.ToBase64String(aes.IV) + "!";
byte[] base64IVBytes = Encoding.UTF8.GetBytes(base64IV);
outputStream.Write(base64IVBytes, 0, base64IVBytes.Length);
// Create a stream which, when we write bytes to it, turns those into
// base64 characters and writes them to outputStream
using var base64Stream = new CryptoStream(outputStream, new ToBase64Transform(), CryptoStreamMode.Write);
// Create a stream which, when we write bytes to it, encrypts them and sends them to
// base64Stream
using var encryptStream = new CryptoStream(base64Stream, aes.CreateEncryptor(), CryptoStreamMode.Write);
// Copy the entirety of our input file into encryptStream. This will encrypt them and
// push them into base64Stream, which will base64-encode them and push them into
// outputStream
inputStream.CopyTo(encryptStream);
Note, that using MD5 to derive key bytes isn't best practice. Use Rfc2898DeriveBytes.
Also note that you don't necessarily need to base64-encode the encrypted result before writing it to a file -- you can just write the encrypted bytes straight out. To go this, get rid of base64Stream, and tell the encryptStream to write straight to outputStream.
Related
Hope someone can help. I have followed a very simple tutorial on how to make an encryption/decryption tool. The encryption works perfectly but the decryption doesn't work. The encryption works in that when i open the encrypted file in a text document it is all random characters but when i try and decrypt it the original data isn't recovered its just more random characters.
private void Btn_Encrypt_Click(object sender, EventArgs e)
{
MessageBox.Show("Your file has been encrypted", "Complete");
// a simle message box used to notify the user that the encryption process has been completed.
Encrypt(ShowPathEnc.Text, TxtSaved.Text, key);
// when the button is clicked the private void "Encrypt" is called and ran.
}
private void Encrypt(string input, string output, string strHash)
{
FileStream inStream, OutStream;
// provideds a stearm for the input(selected raw file) to be ran through the encryption algorithm and for the Output(the saved encryption file) for that file to leave
//the stearm as an encrypted file
CryptoStream CryptStream;
// CryptoStream is used to define a stream that is used to link data streams (inStrean & OutStream) to cryptographyc transformations(ASCII Encoding in this case)
TripleDESCryptoServiceProvider TripCrypto = new TripleDESCryptoServiceProvider();
//Defines a wrapper object to access the cryptographic service provider for triple data encryption
MD5CryptoServiceProvider MD5 = new MD5CryptoServiceProvider();
//defines new object that takes a string and uses MD5 to return a 32 character hexedecimal fotrmated string hash.
byte[] byteHash, byteText;
inStream = new FileStream(input, FileMode.Open, FileAccess.Read);
// takes the raw file that has been selected by the user that is in tne dtream opens that file and reads it.
OutStream = new FileStream(output, FileMode.OpenOrCreate, FileAccess.Write);
// takes the save as file that the user has selected opens or creates it and writes the encrypted data into it
byteHash = MD5.ComputeHash(ASCIIEncoding.ASCII.GetBytes(strHash));
// uses the MD5 hash to compute the data in the file and uses the ASCII key generated earlier to encrypt each byte of data in the file.
byteText = File.ReadAllBytes(input);
//reads all the bytes in the in the raw file making to have every byte encrypted.
MD5.Clear();
// the hash is then cleared for the next encryption as a completely new unique hash will be used for the next file.
TripCrypto.Key = byteHash;
// uses the key to give "TripCrypto" access to the bytes that have been computed by the hash and ASCII encoding.
TripCrypto.Mode = CipherMode.ECB;
//sets the triple layer data encryption to use the CBC standard of encryption.
CryptStream = new CryptoStream(OutStream, TripCrypto.CreateEncryptor(), CryptoStreamMode.Write);
//uses the cyrptostream to take the file in OutStrean tun it through the Triple layer dadta encryption and the the wtite mode to write the encrypted data back to the
//file in the OutStream
int bytesRead;
long length, position = 0;
//instantiates variables used to check the length of the stream in bytes and the position of each byte in the stream
length = inStream.Length;
// gets the length of the stream in bytes
while (position < length)
{
bytesRead = inStream.Read(byteText, 0, byteText.Length);
position += bytesRead;
CryptStream.Write(byteText, 0, bytesRead);
}
//the while loop is comparin g the length of each byte and position of each byte to ensure that there has been a change in position meaning data scrambling
//is used aswell as layered encryption.
inStream.Close();
OutStream.Close();
//files have exited bothe streams and therefore there is no need for those streams to be running and they're closed.
}
private void Btn_Decrypt_Click(object sender, EventArgs e)
{
MessageBox.Show("Your file has been decrypted", "Complete");
// a simle message box used to notify the user that the encryption process has been completed.
decrypt(ShowPathDec.Text, SaveDec.Text, key);
// when the button is clicked the private void "Encrypt" is called and ran.
}
private void decrypt(string input, string output, string strHash)
{
FileStream inStream, OutStream;
// provideds a stearm for the input(selected raw file) to be ran through the encryption algorithm and for the Output(the saved encryption file) for that file to leave
//the stearm as an encrypted file
CryptoStream CryptStream;
// CryptoStream is used to define a stream that is used to link data streams (inStrean & OutStream) to cryptographyc transformations(ASCII Encoding in this case)
TripleDESCryptoServiceProvider TripCrypto = new TripleDESCryptoServiceProvider();
//Defines a wrapper object to access the cryptographic service provider for triple data encryption
MD5CryptoServiceProvider MD5 = new MD5CryptoServiceProvider();
//defines new object that takes a string and uses MD5 to return a 32 character hexedecimal fotrmated string hash.
byte[] byteHash, byteText;
inStream = new FileStream(input, FileMode.Open, FileAccess.Read);
// takes the raw file that has been selected by the user that is in tne dtream opens that file and reads it.
OutStream = new FileStream(output, FileMode.OpenOrCreate, FileAccess.Write);
// takes the save as file that the user has selected opens or creates it and writes the encrypted data into it
byteHash = MD5.ComputeHash(ASCIIEncoding.ASCII.GetBytes(strHash));
// uses the MD5 hash to compute the data in the file and uses the ASCII key generated earlier to encrypt each byte of data in the file.
byteText = File.ReadAllBytes(input);
//reads all the bytes in the in the raw file making to have every byte encrypted.
MD5.Clear();
// the hash is then cleared for the next encryption as a completely new unique hash will be used for the next file.
TripCrypto.Key = byteHash;
// uses the key to give "TripCrypto" access to the bytes that have been computed by the hash and ASCII encoding.
TripCrypto.Mode = CipherMode.ECB;
//sets the triple layer data encryption to use the CBC standard of encryption.
CryptStream = new CryptoStream(OutStream, TripCrypto.CreateDecryptor(), CryptoStreamMode.Write);
//uses the cyrptostream to take the file in OutStrean tun it through the Triple layer dadta encryption and the the wtite mode to write the encrypted data back to the
//file in the OutStream
int bytesRead;
long length, position = 0;
//instantiates variables used to check the length of the stream in bytes and the position of each byte in the stream
length = inStream.Length;
// gets the length of the stream in bytes
while (position < length)
{
bytesRead = inStream.Read(byteText, 0, byteText.Length);
position += bytesRead;
CryptStream.Write(byteText, 0, bytesRead);
}
//the while loop is comparin g the length of each byte and position of each byte to ensure that there has been a change in position meaning data scrambling
//is used aswell as layered encryption.
inStream.Close();
OutStream.Close();
//files have exited bothe streams and therefore there is no need for those streams to be running and they're closed.
}
Short answer: Put a using block around your CryptoStreams.
You're using "Electronic CodeBook" mode for your crypto provider.
This means that it won't write an encrypted block until you've supplied that block with enough data to encrypt.
I don't know the default block length off hand, but let's pretend that it's 4 for illustration purposes.
Text before encryption: ABCDEFGHI
[ABCD] -> Fills up the block and gets encrypted.
[DEFG] -> Fills up the block and gets encrypted.
[HI__] -> Doesn't fill up the code block and never gets encrypted or written to your file.
To force the CryptoStream to encrypt the incomplete block, you can either call CryptoStream.Flush() or you can wrap a using block around your CryptoStream.
I am writing an application, which would receive encrypted byte array, consisting of file name and file bytes, with the following protocol: file_name_and_extension|bytes. Byte array is then decrypted and passing into Encoding.UTF8.getString(decrypted_bytes) would be preferable, because I would like to trim file_name_and_extension from the received bytes to save actual file bytes into file_name_and_extension.
I simplified my application, to only receive file bytes which are then passed into Encoding.UTF8.GetString() and back into byte array with Encoding.UTF8.getBytes(). After that, I am trying to write a zip file, but the file is invalid. It works when using ASCII or Base64.
private void Decryption(byte[] encryptedMessage, byte[] iv)
{
using (Aes aes = new AesCryptoServiceProvider())
{
aes.Key = receiversKey;
aes.IV = iv;
// Decrypt the message
using (MemoryStream decryptedBytes = new MemoryStream())
{
using (CryptoStream cs = new CryptoStream(decryptedBytes, aes.CreateDecryptor(), CryptoStreamMode.Write))
{
cs.Write(encryptedMessage, 0, encryptedMessage.Length);
cs.Close();
string decryptedBytesString = Encoding.UTF8.GetString(decryptedBytes.ToArray()); //corrupts the zip
//string decryptedBytesString = Encoding.ASCII.GetString(decryptedBytes.ToArray()); //works
//String decryptedBytesString = Convert.ToBase64String(decryptedBytes.ToArray()); //works
byte[] fileBytes = Encoding.UTF8.GetBytes(decryptedBytesString);
//byte[] fileBytes = Encoding.ASCII.GetBytes(decryptedBytesString);
//byte[] fileBytes = Convert.FromBase64String(decryptedBytesString);
File.WriteAllBytes("RECEIVED\\received.zip", fileBytes);
}
}
}
}
Because one shouldn't try to interpret raw bytes as symbols in some encoding unless he actually knows/can deduce the encoding used.
If you receive some nonspecific raw bytes, then process them as raw bytes.
But why it works/doesn't work?
Because:
Encoding.Ascii seems to ignore values greater than 127 and return them as they are. So no matter the encoding/decoding done, raw bytes will be the same.
Base64 is a straightforward encoding that won't change the original data in any way.
UTF8 - theoretically with those bytes not being proper UTF8 string we may have some conversion data loss (though it would more likely result in an exception). But the most probable reason is a BOM being added during Encoding.UTF8.GetString call that would remain there after Encoding.UTF8.GetBytes.
In any case, I repeat - do not encode/decode anything unless it is actually string data/required format.
I am in the process of writing a small encoding class which takes a byte[] array, and cyphers it. As the IV can be made public, I wish to prepend it to the ciphertext to use for decryption.
Following the MSDN Documentation and some SO Post source (Cant find link), I started the stream with the raw IV;
//Start encryption process
using (var msEncrypt = new MemoryStream())
{
using (var csEncrypt = new CryptoStream(msEncrypt, encryptor.CreateEncryptor(), CryptoStreamMode.Write))
{
using (var swWriter = new StreamWriter(csEncrypt))
{
//Write all data to the stream.
swWriter.Write(encryptor.IV);
swWriter.Write(plainTextRaw);
}
}
Console.WriteLine("Encrypted Value: {0}", BitConverter.ToString(msEncrypt.ToArray()).Replace("-", String.Empty));
return msEncrypt.ToArray();
}
However, upon writing a simple unit test extracting the first 16 bytes, they do not seem to match.
I am certain the IV is being encrypted within the stream, so where is the best place to inject it?
Just write the IV tot the initial memory stream msEncrypt, not to the stream that is being encrypted swWriter.
Following feedback from Alexei, a simplification of the question:
How do I use a buffered Stream approach to convert the contents of a CryptoStream (using ToBase64Transform) into a StreamWriter (Unicode encoding) without using Convert.ToBase64String()?
Note: Calling Convert.ToBase64String() throws OutOfMemoryException, hence the need for a buffered/Stream approach to the conversion.
You probably should implement custom Stream, not a TextWriter. It is much easier to compose streams than writers (like pass your stream to compressed stream).
To create custom stream - derive from Stream and implement at least Write and Flush (and Read if you need R/W stream). The rest is more or less optional and depends on you additional needs, regular copy to other stream does not need anything else.
In constructor get inner stream passed to you for writing to. Base64 is always producing ASCII characters, so it should be easy to write output as UTF-8 with or without BOM directly to a stream, but if you want to specify encoding you can wrap inner stream with StreamWriter internally.
In your Write implementation buffer data till you get enough bytes to have block of multiple of 3 bytes (i.e. 300) and call Convert.ToBase64String on that portion. Make sure not to loose not-yet-converted portion. Since Base64 converts 3 bytes to 4 characters converting in blocks of multiple of 3 size will never have =/== padding at the end and can be concatenated with next block. So write that converted portion into inner stream/writer. Note that you want to limit block size to something relatively small like 3*10000 to avoid allocation of your blocks on large objects heap.
In Flush make sure to convert the last unwritten bytes (this will be the only one with = padding at the end) and write it to the stream too.
For reading you may need to be more careful as in Base64 white spaces are allowed, so you can't read fixed number of characters and convert to bytes. The easiest approach would be to read by character from StreamReader and convert each 4 non-space ones to bytes.
Note: you can consider writing/reading Base64 by hand directly from bytes. It will give you some performance benefits, but may be hard if you are not good with bit shifting.
Please try using following to encrypt. I am using fileName/filePath as input. You can adjust it as per your requirement. Using this I have encrypted over 1 gb file successfully without any out of memory exception.
public bool EncryptUsingStream(string inputFileName, string outputFileName)
{
bool success = false;
// here assuming that you already have key
byte[] key = new byte[128];
SymmetricAlgorithm algorithm = SymmetricAlgorithm.Create();
algorithm.Key = key;
using (ICryptoTransform transform = algorithm.CreateEncryptor())
{
CryptoStream cs = null;
FileStream fsEncrypted = null;
try
{
using (FileStream fsInput = new FileStream(inputFileName, FileMode.Open, FileAccess.Read))
{
//First write IV
fsEncrypted = new FileStream(outputFileName, FileMode.Create, FileAccess.Write);
fsEncrypted.Write(algorithm.IV, 0, algorithm.IV.Length);
//then write using stream
cs = new CryptoStream(fsEncrypted, transform, CryptoStreamMode.Write);
int bytesRead;
int _bufferSize = 1048576; //buggersize = 1mb;
byte[] buffer = new byte[_bufferSize];
do
{
bytesRead = fsInput.Read(buffer, 0, _bufferSize);
cs.Write(buffer, 0, bytesRead);
} while (bytesRead > 0);
success = true;
}
}
catch (Exception ex)
{
//handle exception or throw.
}
finally
{
if (cs != null)
{
cs.Close();
((IDisposable)cs).Dispose();
if (fsEncrypted != null)
{
fsEncrypted.Close();
}
}
}
}
return success;
}
I need to implement a simple file encryption and then decrypt it, when needed, to a memory stream.
The easiest way seems to do this with File.Encrypt, but is it possible to decrypt the file to memory stream, instead of decrypting the file before reading it to memory stream, and thus exposing it for a while?
And if File.Encrypt is not the best way for this scenario, what would you recommend?
File.Encrypt is an OS feature but it sounds like really you want to control how the encryption is done.
http://msdn.microsoft.com/en-us/library/system.io.file.encrypt.aspx
// This is where the data will be written do.
MemoryStream dataStream = new MemoryStream();
// The encryption vectors
byte[] key = {145,12,32,245,98,132,98,214,6,77,131,44,221,3,9,50};
byte[] iv = {15,122,132,5,93,198,44,31,9,39,241,49,250,188,80,7};
// Build the encryption mathematician
using (TripleDESCryptoServiceProvider encryption = new TripleDESCryptoServiceProvider())
using (ICryptoTransform transform = encryption.CreateEncryptor(key, iv))
using (Stream encryptedOutputStream = new CryptoStream(dataStream, transform, CryptoStreamMode.Write))
using (StreamWriter writer = new StreamWriter(encryptedOutputStream))
{
// In this block, you do your writing, and it will automatically be encrypted
writer.Write("This is the encrypted output data I want to write");
}
Encryption is not for the faint of heart. Be forewarned though, you really should have a strong sense of regular IO and data streams before you attempt this though.
Implementing Crypto deceptively easy, and actually rather tedious, there are a lot of details, and the details wrong are usually what's exploited security wise. The best practice is to use a high level encryption framework that hides these details ivs, salts, mac, comparisons, padding, key rotation, and while it's not improbable for high level frameworks to have the details wrong, when they do, they get found and fixed, the code snippets on stack overflow generally do not.
I have been porting the Google Keyczar framework so such a high level library would exist for C#.
Keyczar-dotnet
And it is usable for encrypting and decrypting io streams.
Install in your project with nuget
PM> Install-Package Keyczar -Pre
Then create your key set. (by having a separate key set file, it gives you the ability to rotating keys in the future and prevents you from accidentally hard coding something that should not ever be hard coded.)
PM> KeyczarTool.exe create --location=path_to_key_set --purpose=crypt
PM> KeyczarTool.exe addkey --location=path_to_key_set --status=primary
Then in your code you can use any IO stream you want for both encryption:
using(var encrypter = new Encrypter("path_to_key_set"))
{
encrypter.Encrypt(plaintextStream, ciphertextStream);
}
and decryption:
using(var crypter = new Crypter("path_to_key_set"))
{
crypter.Decrypt(ciphertextStream, plaintextStream);
}
This was the first encryption code I wrote - be warned, although a good starting point to understand what's going on, static passwords and static salts are a bad idea! (thanks for highlighting this CodesInChaos)
You can decrypt to any stream you like, including straight to a memory stream...
FileInfo file = new FileInfo("SomeFile");
using (FileStream inFs = file.OpenRead())
{
using (MemoryStream outMs = new MemoryStream())
{
encryption.Decrypt(inFs, outMs);
BinaryFormatter bf = new BinaryFormatter();
targetType target= bf.Deserialize(outMs) as targetType;
}
}
where encryption is one of these:
public class EncryptionHelper
{
static SymmetricAlgorithm encryption;
static string password = "password";
static string salt = "this is my salt. There are many like it, but this one is mine.";
static EncryptionHelper()
{
encryption = new RijndaelManaged();
Rfc2898DeriveBytes key = new Rfc2898DeriveBytes(password, Encoding.ASCII.GetBytes(salt));
encryption.Key = key.GetBytes(encryption.KeySize / 8);
encryption.IV = key.GetBytes(encryption.BlockSize / 8);
encryption.Padding = PaddingMode.PKCS7;
}
public void Encrypt(Stream inStream, Stream OutStream)
{
ICryptoTransform encryptor = encryption.CreateEncryptor();
inStream.Position = 0;
CryptoStream encryptStream = new CryptoStream(OutStream, encryptor, CryptoStreamMode.Write);
inStream.CopyTo(encryptStream);
encryptStream.FlushFinalBlock();
}
public void Decrypt(Stream inStream, Stream OutStream)
{
ICryptoTransform encryptor = encryption.CreateDecryptor();
inStream.Position = 0;
CryptoStream encryptStream = new CryptoStream(inStream, encryptor, CryptoStreamMode.Read);
encryptStream.CopyTo(OutStream);
OutStream.Position = 0;
}
}