First some background, in case I'm taking the wrong approach. I have two requirements:
I want to encrypt the data written and read from AnonymousPipeServerStream and AnonymousPipeClientStream
I must use a FIPS-compliant NIST-accredited cryptographic module.
I'm actually using StreamJsonRpc to read and write the pipes, so I have no control over how many bytes are read and written at once. What I'm looking for is an encrypting/decrypting Stream that I can use to wrap unencrypted streams.
I'm trying to use the FIPS-compliant Bouncy Castle .Net library to do this using AES in CTR mode, which I understand is a reasonable way to encrypt a stream.
I can't work out how to increment the counter when encrypting a long data stream, so I get an exception "counter in CTR mode out of range".
Below is a compilable console app which demonstrates what I have so far. It's strongly based on the sample code in section 3.2.2 of the Bouncy Castle BC-FNA user guide.
The code below works, but if you increase the amount of data written to the stream by changing const int COPIES = 50; to const int COPIES = 60;, it will throw an exception.
My questions:
Is this a reasonable approach?
If so, how can I handle large streams?
The code:
using System;
using System.IO;
using System.Text;
using Org.BouncyCastle.Crypto;
using Org.BouncyCastle.Crypto.Fips;
using Org.BouncyCastle.Utilities.Encoders;
namespace Demo
{
class Program
{
static void Main()
{
const int COPIES = 50;
CryptoServicesRegistrar.SetApprovedOnlyMode(true);
var key = new FipsAes.Key(Hex.Decode("aafd12f659cae63489b479e5076ddec2"));
IBlockCipherService provider = CryptoServicesRegistrar.CreateService(key);
// define IV – note 15 bytes, so max message length 255 * 16 bytes
byte[] iv = Hex.Decode("000102030405060708090a0b0c0d0e");
// define data to be encrypted.
string text = "'Twas brillig, and the slithy toves did gyre and gymble in the wabe.\n";
byte[] toBeEncrypted = Encoding.UTF8.GetBytes(text);
// encrypt the data.
var bOut = new MemoryStream();
var encryptorBldr = provider.CreateEncryptorBuilder(FipsAes.Ctr.WithIV(iv));
var encryptor = encryptorBldr.BuildCipher(bOut);
using (Stream encryptingStream = encryptor.Stream)
{
// Write several copies a byte at a time to prove it works.
writeMultipleCopiesOneByteAtOnce(toBeEncrypted, COPIES, encryptingStream); // Change copies to 60 and it breaks.
}
byte[] cipherText = bOut.ToArray();
// decrypt the resulting cipher text
var decryptorBldr = provider.CreateDecryptorBuilder(FipsAes.Ctr.WithIV(iv));
var decryptor = decryptorBldr.BuildCipher(new MemoryStream(cipherText));
using var decIn = decryptor.Stream;
var bytes = readAllOneByteAtOnce(decIn); // Prove that we can read the decryptor stream a byte at a time.
string result = Encoding.UTF8.GetString(bytes); // Print decrypted text.
Console.WriteLine(result);
}
// Just for test purposes. I wouldn't normally write bytes like this.
static void writeMultipleCopiesOneByteAtOnce(byte[] data, int copies, Stream output)
{
for (int i = 0; i < copies; ++i)
{
foreach (byte b in data)
output.WriteByte(b);
}
}
// Just for test purposes. I wouldn't normally read bytes like this.
static byte[] readAllOneByteAtOnce(Stream stream)
{
using var memStream = new MemoryStream();
while (true)
{
int b = stream.ReadByte();
if (b < 0)
return memStream.ToArray();
memStream.WriteByte((byte)b);
}
}
}
}
Related
Consider the following code (you also may check in sandbox):
using System;
using System.IO;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
class EncryptionIVTest
{
private static readonly string Data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
private static readonly byte[] Password = Guid.NewGuid().ToByteArray().Take(32).ToArray();
static void Main()
{
var iv = Guid.NewGuid().ToByteArray().Take(16).ToArray(); // random initialization vector
var iv2 = new byte[16]; // just another zero-filled initialization vector
var encrypted = Encrypt(iv);
Console.WriteLine($"Original: {Data}");
Console.WriteLine($"Encrypted: {encrypted}");
Console.WriteLine($"Decrypted: {Decrypt(encrypted, iv)}");
Console.WriteLine($"Decrypted with another IV: {Decrypt(encrypted, iv2)}"); // It should throw exception or output completely mangled string
}
private static string Encrypt(byte[] iv)
{
var cipher = CreateCipher(iv);
var buf = Encoding.UTF8.GetBytes(Data);
using var ms = new MemoryStream();
using (var stream = new CryptoStream(ms, cipher.CreateEncryptor(), CryptoStreamMode.Write))
stream.Write(buf, 0, buf.Length);
return Convert.ToBase64String(ms.ToArray());
}
private static string Decrypt(string encrypted, byte[] iv)
{
var cipher = CreateCipher(iv);
using var ms = new MemoryStream(Convert.FromBase64String(encrypted));
using var result = new MemoryStream();
using (var stream = new CryptoStream(ms, cipher.CreateDecryptor(), CryptoStreamMode.Read))
stream.CopyTo(result);
return Encoding.UTF8.GetString(result.GetBuffer(), 0, (int)result.Length);
}
private static Aes CreateCipher(byte[] iv)
{
var cipher = Aes.Create();
cipher.Key = Password;
cipher.IV = iv;
cipher.Mode = CipherMode.CBC;
return cipher;
}
}
It outputs:
Original: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Encrypted: EROKh8lVgREvTqzBYXjEm7EbTIT883uR9wsD82lRM14KtiOYr+/+ZpAwz/UfprqSP5mIQ7Du/d43Y88hAPjvkA==
Decrypted: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Decrypted with another IV: ???#?n? ??7║??Paaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
The fourth string is not fully mangled, it contains an untouched trailing. It seems like encryptor only mangle first 16 bytes (size of an initialization vector) and leaves other untouched. By default, encryptor uses CBC CipherMode and it should mangle all data if I understand correctly.
Is it possible to mangle all data, not only the first part?
The purpose of the IV is not to "mangle" data further or to serve as a second encryption key - that would just make it redundant of the actual key.
The purpose is to provide additional entropy so that two sets of plaintext data encrypted with the same key but with different IVs will appear completely different when encrypted. This makes it harder for an attacker to infer anything about the data. For example, without the IV, sophisticated attackers could run statistical analyses based on language patterns and potentially figure out what certain encrypted packets actually are based on how frequently they occur.
So what you're seeing should not be surprising or concerning. The IV is doing its job.
By the way, using a Guid as a key is NOT secure. First of all it's only 16-bytes not 32 so you only have a 128-bit key basically. See https://learn.microsoft.com/en-us/dotnet/standard/security/generating-keys-for-encryption-and-decryption#symmetric-keys for the right apis to use to generate keys and IVs
I have a working implementation of TripleDESCng (tested against some test vectors), but the following happens:
When I encrypt plain text This is a sample message (24 bytes, thus for this it would be 3 blocks) (hex for it is 5468697320697320612073616D706C65206D657373616765) with an example key, I get E81F113DD7C5D965E082F3D42EC1E2CA39BCDBCCBC0A2BD9. However, when I decrypt this with the same example key, I get 5468697320697320612073616D706C650000000000000000, which, when converted back to ASCII, is:
This is a sample.
Any reason other than my code why this would behave this way? To encrypt and decrypt, I use 24 byte keys (ECB mode).
EDIT:
using (var tripleDES = new TripleDESCryptoServiceProvider())
{
byte[] data = ASCIIEncoding.ASCII.GetBytes("This is a sample message");
Console.WriteLine(BitConverter.ToString(data));
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
encryptor.TransformBlock(data, 0, data.Length, result, 0);
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[result.Length];
decryptor.TransformBlock(result, 0, result.Length, result2, 0);
Console.WriteLine(BitConverter.ToString(result2));
}
Console.ReadLine();
With almost all modes1, you should make sure that the final part of your data is pushed through TransformFinalBlock rather than TransformBlock2, to make sure it knows that no more data is coming and to ensure final blocks are flushed/written.
It's bad form, in general, to assume the output size is going to match the input size.
the mode is not a problem, IV is set to 0s either way
Yes, that'll mean that the first block was not affected by your choice of Mode. But all subsequent blocks will be, because they will use the chaining mode and the previous block, not the IV. So if you want ECB (you shouldn't3) you need to explicitly set that mode.
1Your code is using CBC, not EBC as you claim in your narrative. CBC is the default mode for .NET encryption classes.
2And when using this second method, pay attention to it's return value, as mjwills commented.
3You've picked an outdated crypto algorithm, paired it with an outdated mode of operation, and your words I've quoted above mean that you don't understand modes. Added together, I would suggest that you're not well placed to be writing code that uses crypto currently. The .NET classes can make it seem easy to write crypto code but you still have to understand how to make good choices in using them. Better to spend more time on researching these things before writing code.
I think that your problem is in the method of the encryptor / decryptor that you are using: the TransformBlock method is conceived to transform a block when you will be encrypting multiple blocks.
That is not the case in your code, where you want to transform a single block, and therefore you should be using the TransformFinalBlock method instead. BTW I took the liberty of making your sample buildable.
using System;
using System.Text;
namespace Tests
{
class Program
{
static void Main(string[] args)
{
System.Security.Cryptography.TripleDESCryptoServiceProvider tripleDES = new System.Security.Cryptography.TripleDESCryptoServiceProvider();
byte[] data = Encoding.UTF8.GetBytes("This is a sample message");
byte[] key = Encoding.UTF8.GetBytes("NOSTROMOHASSOMEGODPOWERS");
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
result = encryptor.TransformFinalBlock(data, 0, data.Length);
string res = BitConverter.ToString(result).Replace("-","");
Console.WriteLine(BitConverter.ToString(result).Replace("-",""));
byte[] data2 = result;
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[data2.Length];
result2 = decryptor.TransformFinalBlock(data2, 0, data2.Length);
Console.WriteLine(Encoding.UTF8.GetString(result2));
}
}
}
I have a working implementation of TripleDESCng (tested against some test vectors), but the following happens:
When I encrypt plain text This is a sample message (24 bytes, thus for this it would be 3 blocks) (hex for it is 5468697320697320612073616D706C65206D657373616765) with an example key, I get E81F113DD7C5D965E082F3D42EC1E2CA39BCDBCCBC0A2BD9. However, when I decrypt this with the same example key, I get 5468697320697320612073616D706C650000000000000000, which, when converted back to ASCII, is:
This is a sample.
Any reason other than my code why this would behave this way? To encrypt and decrypt, I use 24 byte keys (ECB mode).
EDIT:
using (var tripleDES = new TripleDESCryptoServiceProvider())
{
byte[] data = ASCIIEncoding.ASCII.GetBytes("This is a sample message");
Console.WriteLine(BitConverter.ToString(data));
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
encryptor.TransformBlock(data, 0, data.Length, result, 0);
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[result.Length];
decryptor.TransformBlock(result, 0, result.Length, result2, 0);
Console.WriteLine(BitConverter.ToString(result2));
}
Console.ReadLine();
With almost all modes1, you should make sure that the final part of your data is pushed through TransformFinalBlock rather than TransformBlock2, to make sure it knows that no more data is coming and to ensure final blocks are flushed/written.
It's bad form, in general, to assume the output size is going to match the input size.
the mode is not a problem, IV is set to 0s either way
Yes, that'll mean that the first block was not affected by your choice of Mode. But all subsequent blocks will be, because they will use the chaining mode and the previous block, not the IV. So if you want ECB (you shouldn't3) you need to explicitly set that mode.
1Your code is using CBC, not EBC as you claim in your narrative. CBC is the default mode for .NET encryption classes.
2And when using this second method, pay attention to it's return value, as mjwills commented.
3You've picked an outdated crypto algorithm, paired it with an outdated mode of operation, and your words I've quoted above mean that you don't understand modes. Added together, I would suggest that you're not well placed to be writing code that uses crypto currently. The .NET classes can make it seem easy to write crypto code but you still have to understand how to make good choices in using them. Better to spend more time on researching these things before writing code.
I think that your problem is in the method of the encryptor / decryptor that you are using: the TransformBlock method is conceived to transform a block when you will be encrypting multiple blocks.
That is not the case in your code, where you want to transform a single block, and therefore you should be using the TransformFinalBlock method instead. BTW I took the liberty of making your sample buildable.
using System;
using System.Text;
namespace Tests
{
class Program
{
static void Main(string[] args)
{
System.Security.Cryptography.TripleDESCryptoServiceProvider tripleDES = new System.Security.Cryptography.TripleDESCryptoServiceProvider();
byte[] data = Encoding.UTF8.GetBytes("This is a sample message");
byte[] key = Encoding.UTF8.GetBytes("NOSTROMOHASSOMEGODPOWERS");
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
result = encryptor.TransformFinalBlock(data, 0, data.Length);
string res = BitConverter.ToString(result).Replace("-","");
Console.WriteLine(BitConverter.ToString(result).Replace("-",""));
byte[] data2 = result;
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[data2.Length];
result2 = decryptor.TransformFinalBlock(data2, 0, data2.Length);
Console.WriteLine(Encoding.UTF8.GetString(result2));
}
}
}
so i using this code to encrypt my file
as you can see iam using public PGP
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.2.6 (GNU/Linux)
masdSFVkRBADYxPZYC+nu9nhSVkxcVkVJ5axZKzCRuygqUxka
kZIBy2CAQVKz5dBkRaUkaaksbcyautks7asaov26Fc9cT25Rvnh7
wYIJhcRoIl4cxashdgutasd0qfcOnVB5JVCQDhXclBW7kwCgkoUW
....
...
...
-----END PGP PUBLIC KEY BLOCK-----
the code works fine but i think the data of the encrepted file is corrupted
because it doesnt comes out in this format (like the key)
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.2.6 (GNU/Linux)
masdSFVkRBADYxPZYC+nu9nhSVkxcVkVJ5axZKzCRuygqUxka
kZIBy2CAQVKz5dBkRaUkaaksbcyautks7asaov26Fc9cT25Rvnh7
wYIJhcRoIl4cxashdgutasd0qfcOnVB5JVCQDhXclBW7kwCgkoUW
....
...
...
-----END PGP PUBLIC KEY BLOCK-----
am i wrong?
dont the output should be in the same format ?
using System;
using System.Xml;
using System.IO;
using System.Security.Cryptography;
using System.Security.Cryptography.Xml;
using System.Text;
using Org.BouncyCastle.Bcpg.OpenPgp;
using Org.BouncyCastle.Security;
using Org.BouncyCastle.Utilities.IO;
using Org.BouncyCastle.Utilities.Encoders;
using Org.BouncyCastle.Bcpg;
class CPGPencrypt
{
private static PgpPublicKey ReadPublicKey(Stream inputStream)
{
inputStream = PgpUtilities.GetDecoderStream(inputStream);
PgpPublicKeyRingBundle pgpPub = new PgpPublicKeyRingBundle(inputStream);
//
// we just loop through the collection till we find a key suitable for encryption, in the real
// world you would probably want to be a bit smarter about this.
//
//
// iterate through the key rings.
//
foreach (PgpPublicKeyRing kRing in pgpPub.GetKeyRings())
{
foreach (PgpPublicKey k in kRing.GetPublicKeys())
{
if (k.IsEncryptionKey)
{
return k;
}
}
}
throw new ArgumentException("Can't find encryption key in key ring.");
}
private static byte[] EncryptFile(byte[] clearData, string fileName, PgpPublicKey encKey, bool withIntegrityCheck)
{
MemoryStream bOut = new MemoryStream();
PgpCompressedDataGenerator comData = new PgpCompressedDataGenerator(
CompressionAlgorithmTag.Zip);
Stream cos = comData.Open(bOut); // open it with the final destination
PgpLiteralDataGenerator lData = new PgpLiteralDataGenerator();
// we want to Generate compressed data. This might be a user option later,
// in which case we would pass in bOut.
Stream pOut = lData.Open(
cos, // the compressed output stream
PgpLiteralData.Binary,
fileName, // "filename" to store
clearData.Length, // length of clear data
DateTime.UtcNow // current time
);
pOut.Write(clearData, 0, clearData.Length);
lData.Close();
comData.Close();
PgpEncryptedDataGenerator cPk = new PgpEncryptedDataGenerator(SymmetricKeyAlgorithmTag.Cast5, new SecureRandom());
cPk.AddMethod(encKey);
byte[] bytes = bOut.ToArray();
MemoryStream encOut = new MemoryStream();
Stream os = encOut;
Stream cOut = cPk.Open(os, bytes.Length);
cOut.Write(bytes, 0, bytes.Length); // obtain the actual bytes from the compressed stream
cOut.Close();
encOut.Close();
return encOut.ToArray();
}
public static string Encrypt(string file_name,string file_to_read)
{
try
{
byte[] dataBytes = File.ReadAllBytes(file_to_read);
Stream keyIn = File.OpenRead("pgpdata-public.asc");
Stream outStream = File.Create(#"myfolder\"+file_name);
byte[] encrypted = EncryptFile(dataBytes, #"myfolder\"+file_name, ReadPublicKey(keyIn), false);
outStream.Write(encrypted, 0, encrypted.Length);
keyIn.Close();
outStream.Close();
}
catch (Exception e)
{
return e.Message;
}
return file_name;
}
}
There are different encoding schemes in OpenPGP, namely
binary data and
ASCII armored data.
Especially for key exchange, normally the ASCII armored format is preferred as it is more robust and easy to recognize. For mail exchange, it is mandatory (for 7 bit compatibility). The binary version also has advantages, especially regarding performance and storage (bandwith) requirements.
For example, GnuPG will use the binary encoding by default, unless you request the ASCII armored version using the option --ascii or abbreviated -a.
It look like your code is outputting the binary encoding, but works all fine.
You can easily test by trying to decrypt (eg. using GnuPG: gpg --decrypt file.pgp). Alternatively, you can dump the OpenPGP packets the file contains by using gpg --list-packets file.pgp or using the more verbose utility pgpdump, which is available in most (unix) package repositories: pgpdump file.pgp. Unlike gpg --list-packets, it also resolves packet and algorithm identifiers to human readable strings (where gpg --list-packets just dumps their numeric IDs).
My text file says The quick brown fox jumps over the lazy dog, however when i try to get the hash from this file both the md5 and sha1 is different from the wikipedias result. I have 3 questions. 1) What did i do wrong in the code? 2) How can i have this piece of code better? (do i need the Initialize) 3) How do i salt this?
{
const int bufSize = 1024 * 8;
int read;
byte[] buf = new byte[bufSize];
string fn = #"b.txt";
byte[] result1 = new byte[0];
byte[] result2 = new byte[0];
SHA1 sha = new SHA1CryptoServiceProvider();
MD5 md5 = new MD5CryptoServiceProvider();
sha.Initialize();
md5.Initialize();
FileStream fin = File.OpenRead(fn);
while ((read = fin.Read(buf, 0, buf.Length)) != 0)
{
result1 = sha.ComputeHash(buf);
result2 = md5.ComputeHash(buf);
}
fin.Close();
MessageBox.Show(myFunc(result1));
MessageBox.Show(myFunc(result2));
}
(EDIT: Disposing of the hash algorithms now. I suspect it's unnecessary, but it's good practice :)
You're calling ComputeHash for the whole buffer even though you should only be hashing the portion of the buffer you've read. In addition, you're computing a new hash for each call to Read.
Here's some really simple code to compute the hashes:
using System;
using System.IO;
using System.Security.Cryptography;
class Test
{
static void Main()
{
byte[] plaintext = File.ReadAllBytes("b.txt");
using (MD5 md5 = MD5.Create())
{
byte[] md5Hash = md5.ComputeHash(plaintext);
Console.WriteLine(BitConverter.ToString(md5Hash));
}
using (SHA1 sha1 = SHA1.Create())
{
byte[] sha1Hash = sha1.ComputeHash(plaintext);
Console.WriteLine(BitConverter.ToString(sha1Hash));
}
}
}
This gives the results as per wikipedia - note that b.txt shouldn't have a newline at the end of it.
An alternative way of getting the binary data to start with would be:
byte[] plaintext = Encoding.ASCII.GetBytes(
"The quick brown fox jumps over the lazy dog");
Note that this is just the simple way of computing a hash in one go. If you want to do it in a streaming fashion (i.e. where you read some data, add it to the hash, read some more data etc) then either you can use the ComputeHash(Stream) overload or (if you want to "push" data to it) you can use TransformBlock and TransformFinalBlock, like this:
using System.Text;
class Test
{
static void Main()
{
using (MD5 md5 = MD5.Create())
using (SHA1 sha1 = SHA1.Create())
using (Stream input = File.OpenRead("b.txt"))
{
// Artificially small to make sure there's
// more than one read
byte[] buffer = new byte[4];
int bytesRead;
while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
{
md5.TransformBlock(buffer, 0, bytesRead, null, 0);
sha1.TransformBlock(buffer, 0, bytesRead, null, 0);
}
md5.TransformFinalBlock(buffer, 0, 0);
sha1.TransformFinalBlock(buffer, 0, 0);
Console.WriteLine(BitConverter.ToString(md5.Hash));
Console.WriteLine(BitConverter.ToString(sha1.Hash));
}
}
}
Note the way we pass null to TransformBlock because we don't need any output, and we don't transform any data in the final block. I suspect this is the example you'll want to use, based on your previous comments.
Regarding different results, it could depend on the character encoding you are using in your text, since these hash algorithms deal with bytes.
As far as improving your code, you are not disposing of your CryptoServiceProviders. Everything that inherits from HashAlgorithm implements IDisposable and needs to be disposed when you're finished with it, either by using using() or calling Dispose() directly. You're also not disposing of the FileStream, which has the same requirements regarding IDisposable.