Develop Reference

C# Out of Memory while handling and encrypting large files

Im trying to encrypt a large file (Camtasia.exe) with the AES encryption.
Now for some reason I get a "Out of Memory" Exception. Im really new to this and I don't know how I could possibly fix that. This is my code
I use this to call my encryption method.
bytes = File.ReadAllBytes("Camtasia.exe");
Cryptography.Encryption.EncryptAES(System.Text.Encoding.Default.GetString(bytes), encryptionKey);
This is the AES encryption itself
public static string EncryptAES(string content, string password)
{
byte[] bytes = Encoding.UTF8.GetBytes(content);
using (SymmetricAlgorithm crypt = Aes.Create())
using (HashAlgorithm hash = MD5.Create())
using (MemoryStream memoryStream = new MemoryStream())
{
crypt.Key = hash.ComputeHash(Encoding.UTF8.GetBytes(password));
// This is really only needed before you call CreateEncryptor the second time,
// since it starts out random. But it's here just to show it exists.
crypt.GenerateIV();
using (CryptoStream cryptoStream = new CryptoStream(
memoryStream, crypt.CreateEncryptor(), CryptoStreamMode.Write))
{
cryptoStream.Write(bytes, 0, bytes.Length);
}
string base64IV = Convert.ToBase64String(crypt.IV);
string base64Ciphertext = Convert.ToBase64String(memoryStream.ToArray());
return base64IV + "!" + base64Ciphertext;
}
}
Here is the error again that I get when calling the function "EncryptAES" at the top. I would be glad if someone could explain how this happens and how to solve it
https://imgur.com/xqcLsKW

You're reading the entire exe into memory, interpreting it as a UTF-16 string (??!), turning that back into UTF-8 bytes, and encrypting those. This converting to/from a string is horifically broken. An executable file is not a human-readable string, and even if it was, you're in a real muddle as to which encoding you're using. I think you can drop the whole string thing.
You're also reading the entire thing into memory (several times in fact, because of the whole string thing), which is wasteful. You don't need to do that: you can encrypt it bit-by-bit. To do this, use a Stream.
Something like this should work (untested): at least it gets the general concept across. We set up a series of streams which lets us read the data out of the input file bit-by-bit, and write them out to the output file bit-by-bit.
// The file we're reading from
using var inputStream = File.OpenRead("Camtasia.exe");
// The file we're writing to
using var outputStream = File.OpenWrite("EncryptedFile.txt");
using var HashAlgorithm hash = MD5.Create();
using var aes = Aes.Create();
aes.Key = hash.ComputeHash(Encoding.UTF8.GetBytes(password));
// Turn the IV into a base64 string, add "!", encode as UTF-8, and write to the file
string base64IV = Convert.ToBase64String(aes.IV) + "!";
byte[] base64IVBytes = Encoding.UTF8.GetBytes(base64IV);
outputStream.Write(base64IVBytes, 0, base64IVBytes.Length);
// Create a stream which, when we write bytes to it, turns those into
// base64 characters and writes them to outputStream
using var base64Stream = new CryptoStream(outputStream, new ToBase64Transform(), CryptoStreamMode.Write);
// Create a stream which, when we write bytes to it, encrypts them and sends them to
// base64Stream
using var encryptStream = new CryptoStream(base64Stream, aes.CreateEncryptor(), CryptoStreamMode.Write);
// Copy the entirety of our input file into encryptStream. This will encrypt them and
// push them into base64Stream, which will base64-encode them and push them into
// outputStream
inputStream.CopyTo(encryptStream);
Note, that using MD5 to derive key bytes isn't best practice. Use Rfc2898DeriveBytes.
Also note that you don't necessarily need to base64-encode the encrypted result before writing it to a file -- you can just write the encrypted bytes straight out. To go this, get rid of base64Stream, and tell the encryptStream to write straight to outputStream.

Binary Serialization in c#

I am trying to serialize a class object using binary serialization in C#. I have tried and everywhere all I can find that the serialized data goes to a file always in all the examples I have seen.
In my case, I have to store the serialized data in SQL. The following is an example of the method I have created.
//Serializing the List
public void Serialize(Employees emps, String filename)
{
//Create the stream to add object into it.
System.IO.Stream ms = File.OpenWrite(filename);
//Format the object as Binary
BinaryFormatter formatter = new BinaryFormatter();
//It serialize the employee object
formatter.Serialize(ms, emps);
ms.Flush();
ms.Close();
ms.Dispose();
}
How can I get the serialized data directly in a string variable? I don't want to use a file.
Please help.

The easiest way to represent a byte array as a string in C# is with base64 encoding. The below example shows how this would be achieved within your code.
public void Serialize(Employees emps, String filename)
{
//Create the stream to add object into it.
MemoryStream ms = new MemoryStream();
//Format the object as Binary
BinaryFormatter formatter = new BinaryFormatter();
//It serialize the employee object
formatter.Serialize(ms, emps);
// Your employees object serialised and converted to a string.
string encodedObject = Convert.ToBase64String(ms.ToArray());
ms.Close();
}
This creates the string encodedObject. To retrieve the byte array and your serialised object back from the string you will use the below code.
BinaryFormatter bf = new BinaryFormatter();
// Decode the string back to a byte array
byte[] decodedObject = Convert.FromBase64String(encodedObject);
// Create a memory stream and pass in the decoded byte array as the parameter
MemoryStream memoryStream = new MemoryStream(decodedObject);
// Deserialise byte array back to employees object.
Employees employees = bf.Deserialize(memoryStream);

Just use MemoryStream ms = new MemoryStream() instead of your file stream.
You can extract a byte[] for Storage to SQL after serializing by calling ms.ToArray().
And don't forget to put your Stream into a using-Statement, to guarantee correct disposal of the allocated resources.

Byte array read from a file and byte array converted from string read from same file differs

If i read byte array from a file and write it using below code
byte[] bytes = File.ReadAllBytes(filePath);
File.WriteAllBytes(filePath, byteArr);
works perfectly fine.I can open and view the written file properly.
But if i read file contents into a string and then convert it to byte array using below function
string s = File.ReadAllText(filePath);
var byteArr = System.Text.Encoding.UTF8.GetBytes(s);
the size of byte array is more than the previous array read directly from file and the values are also different, hence if i write the file using this array the cannot be read when opened
Note:- File is utf-8 encoded
i found out that using below code
using (StreamReader reader = new StreamReader(filePath, Encoding.UTF8, true))
{
reader.Peek(); // you need this!
var encoding = reader.CurrentEncoding;
}
Unable to understand why both the array differs??
I was using the below attached image for converting and then writing

With
using (StreamReader reader = new StreamReader(filePath, Encoding.UTF8, true))
{
reader.Peek(); // you need this!
var encoding = reader.CurrentEncoding;
}
your var encoding will just echo the Encoding.UTF8 parameter. You are deceiving yourself there.
A binary file just has no text encoding.
Need to save a file may be anything an image or a text
Then just use ReadAllBytes/WriteAllBytes. A text file is always also a byte[], but not all file types are text. You would need Base64 encoding first and that just adds to the size.

The safest way to convert byte arrays to strings is indeed encoding it in something like base64.
Like:
string s= Convert.ToBase64String(bytes);
byte[] bytes = Convert.FromBase64String(s);

Why does UTF8 encoding change/corrupt bytes as oppose to Base64 and ASCII, when writing to file?

I am writing an application, which would receive encrypted byte array, consisting of file name and file bytes, with the following protocol: file_name_and_extension|bytes. Byte array is then decrypted and passing into Encoding.UTF8.getString(decrypted_bytes) would be preferable, because I would like to trim file_name_and_extension from the received bytes to save actual file bytes into file_name_and_extension.
I simplified my application, to only receive file bytes which are then passed into Encoding.UTF8.GetString() and back into byte array with Encoding.UTF8.getBytes(). After that, I am trying to write a zip file, but the file is invalid. It works when using ASCII or Base64.
private void Decryption(byte[] encryptedMessage, byte[] iv)
{
using (Aes aes = new AesCryptoServiceProvider())
{
aes.Key = receiversKey;
aes.IV = iv;
// Decrypt the message
using (MemoryStream decryptedBytes = new MemoryStream())
{
using (CryptoStream cs = new CryptoStream(decryptedBytes, aes.CreateDecryptor(), CryptoStreamMode.Write))
{
cs.Write(encryptedMessage, 0, encryptedMessage.Length);
cs.Close();
string decryptedBytesString = Encoding.UTF8.GetString(decryptedBytes.ToArray()); //corrupts the zip
//string decryptedBytesString = Encoding.ASCII.GetString(decryptedBytes.ToArray()); //works
//String decryptedBytesString = Convert.ToBase64String(decryptedBytes.ToArray()); //works
byte[] fileBytes = Encoding.UTF8.GetBytes(decryptedBytesString);
//byte[] fileBytes = Encoding.ASCII.GetBytes(decryptedBytesString);
//byte[] fileBytes = Convert.FromBase64String(decryptedBytesString);
File.WriteAllBytes("RECEIVED\\received.zip", fileBytes);
}
}
}
}

Because one shouldn't try to interpret raw bytes as symbols in some encoding unless he actually knows/can deduce the encoding used.
If you receive some nonspecific raw bytes, then process them as raw bytes.
But why it works/doesn't work?
Because:
Encoding.Ascii seems to ignore values greater than 127 and return them as they are. So no matter the encoding/decoding done, raw bytes will be the same.
Base64 is a straightforward encoding that won't change the original data in any way.
UTF8 - theoretically with those bytes not being proper UTF8 string we may have some conversion data loss (though it would more likely result in an exception). But the most probable reason is a BOM being added during Encoding.UTF8.GetString call that would remain there after Encoding.UTF8.GetBytes.
In any case, I repeat - do not encode/decode anything unless it is actually string data/required format.

How to insert List<> values into SQL Binary field using C#

I'm not entirely new to programming but I still see myself as a novice. I'm currently creating an Invoicing system with a max of 5 line items, this being said, I'm creating a String<> item, serializing it to store and then de-serializing it to display.
So far I've managed the serializing, and de-serializing, and from the de-serialized value I've managed to display the relevant information in the correct fields.
My question comes to: HOW do I add the list of items in the String<> object to either a Binary or XML field in my SQL table?
I know it should be similar to adding an Image object to binary but there's a catch there. usually:
byte[] convertToByte(string sourcePath)
{
//get the byte file size of image
FileInfo fInfo = new FileInfo(sourcePath);
long byteSize = fInfo.Length;
//read the file using file stream
FileStream fStream = new FileStream(sourcePath, FileMode.Open, FileAccess.Read);
//read again as byte using binary reader
BinaryReader binRead = new BinaryReader(fStream);
//convert image to byte (already)
byte[] data = binRead.ReadBytes((int)byteSize);
return data;
}
this kind of thing is done for an image however the whole "long" thing does not apply to the List<> object.
Any assistance would be helpful

If you simply want to store your data as "readable" text, you can use the varchar(MAX) or nvarchar(MAX) (depending on whether you need extended character support). That translates directly into a string in ADO.NET or EntityFramework.
If all you need are bytes from a string, the Encoding class will do that:
System.Text.Encoding.Default.GetBytes(yourstring);
See: http://msdn.microsoft.com/en-us/library/ds4kkd55%28v=vs.110%29.aspx

A way of saving a binary file in a string is to convert the image to a Base64 string. This can be done with the Convert.ToBase64String (Byte[]) method:
Convert.ToBase64String msdn
string convertImageToBase64(string sourcePath)
{
//get the byte file size of image
FileInfo fInfo = new FileInfo(sourcePath);
long byteSize = fInfo.Length;
//read the file using file stream
FileStream fStream = new FileStream(sourcePath, FileMode.Open, FileAccess.Read);
//read again as byte using binary reader
BinaryReader binRead = new BinaryReader(fStream);
//convert image to byte (already)
byte[] data = binRead.ReadBytes((int)byteSize);
return Convert.ToBase64String (data);
}
Now you will be able to save the Base64 string in a string field in your database.