My text file says The quick brown fox jumps over the lazy dog, however when i try to get the hash from this file both the md5 and sha1 is different from the wikipedias result. I have 3 questions. 1) What did i do wrong in the code? 2) How can i have this piece of code better? (do i need the Initialize) 3) How do i salt this?
{
const int bufSize = 1024 * 8;
int read;
byte[] buf = new byte[bufSize];
string fn = #"b.txt";
byte[] result1 = new byte[0];
byte[] result2 = new byte[0];
SHA1 sha = new SHA1CryptoServiceProvider();
MD5 md5 = new MD5CryptoServiceProvider();
sha.Initialize();
md5.Initialize();
FileStream fin = File.OpenRead(fn);
while ((read = fin.Read(buf, 0, buf.Length)) != 0)
{
result1 = sha.ComputeHash(buf);
result2 = md5.ComputeHash(buf);
}
fin.Close();
MessageBox.Show(myFunc(result1));
MessageBox.Show(myFunc(result2));
}
(EDIT: Disposing of the hash algorithms now. I suspect it's unnecessary, but it's good practice :)
You're calling ComputeHash for the whole buffer even though you should only be hashing the portion of the buffer you've read. In addition, you're computing a new hash for each call to Read.
Here's some really simple code to compute the hashes:
using System;
using System.IO;
using System.Security.Cryptography;
class Test
{
static void Main()
{
byte[] plaintext = File.ReadAllBytes("b.txt");
using (MD5 md5 = MD5.Create())
{
byte[] md5Hash = md5.ComputeHash(plaintext);
Console.WriteLine(BitConverter.ToString(md5Hash));
}
using (SHA1 sha1 = SHA1.Create())
{
byte[] sha1Hash = sha1.ComputeHash(plaintext);
Console.WriteLine(BitConverter.ToString(sha1Hash));
}
}
}
This gives the results as per wikipedia - note that b.txt shouldn't have a newline at the end of it.
An alternative way of getting the binary data to start with would be:
byte[] plaintext = Encoding.ASCII.GetBytes(
"The quick brown fox jumps over the lazy dog");
Note that this is just the simple way of computing a hash in one go. If you want to do it in a streaming fashion (i.e. where you read some data, add it to the hash, read some more data etc) then either you can use the ComputeHash(Stream) overload or (if you want to "push" data to it) you can use TransformBlock and TransformFinalBlock, like this:
using System.Text;
class Test
{
static void Main()
{
using (MD5 md5 = MD5.Create())
using (SHA1 sha1 = SHA1.Create())
using (Stream input = File.OpenRead("b.txt"))
{
// Artificially small to make sure there's
// more than one read
byte[] buffer = new byte[4];
int bytesRead;
while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
{
md5.TransformBlock(buffer, 0, bytesRead, null, 0);
sha1.TransformBlock(buffer, 0, bytesRead, null, 0);
}
md5.TransformFinalBlock(buffer, 0, 0);
sha1.TransformFinalBlock(buffer, 0, 0);
Console.WriteLine(BitConverter.ToString(md5.Hash));
Console.WriteLine(BitConverter.ToString(sha1.Hash));
}
}
}
Note the way we pass null to TransformBlock because we don't need any output, and we don't transform any data in the final block. I suspect this is the example you'll want to use, based on your previous comments.
Regarding different results, it could depend on the character encoding you are using in your text, since these hash algorithms deal with bytes.
As far as improving your code, you are not disposing of your CryptoServiceProviders. Everything that inherits from HashAlgorithm implements IDisposable and needs to be disposed when you're finished with it, either by using using() or calling Dispose() directly. You're also not disposing of the FileStream, which has the same requirements regarding IDisposable.
Related
First some background, in case I'm taking the wrong approach. I have two requirements:
I want to encrypt the data written and read from AnonymousPipeServerStream and AnonymousPipeClientStream
I must use a FIPS-compliant NIST-accredited cryptographic module.
I'm actually using StreamJsonRpc to read and write the pipes, so I have no control over how many bytes are read and written at once. What I'm looking for is an encrypting/decrypting Stream that I can use to wrap unencrypted streams.
I'm trying to use the FIPS-compliant Bouncy Castle .Net library to do this using AES in CTR mode, which I understand is a reasonable way to encrypt a stream.
I can't work out how to increment the counter when encrypting a long data stream, so I get an exception "counter in CTR mode out of range".
Below is a compilable console app which demonstrates what I have so far. It's strongly based on the sample code in section 3.2.2 of the Bouncy Castle BC-FNA user guide.
The code below works, but if you increase the amount of data written to the stream by changing const int COPIES = 50; to const int COPIES = 60;, it will throw an exception.
My questions:
Is this a reasonable approach?
If so, how can I handle large streams?
The code:
using System;
using System.IO;
using System.Text;
using Org.BouncyCastle.Crypto;
using Org.BouncyCastle.Crypto.Fips;
using Org.BouncyCastle.Utilities.Encoders;
namespace Demo
{
class Program
{
static void Main()
{
const int COPIES = 50;
CryptoServicesRegistrar.SetApprovedOnlyMode(true);
var key = new FipsAes.Key(Hex.Decode("aafd12f659cae63489b479e5076ddec2"));
IBlockCipherService provider = CryptoServicesRegistrar.CreateService(key);
// define IV – note 15 bytes, so max message length 255 * 16 bytes
byte[] iv = Hex.Decode("000102030405060708090a0b0c0d0e");
// define data to be encrypted.
string text = "'Twas brillig, and the slithy toves did gyre and gymble in the wabe.\n";
byte[] toBeEncrypted = Encoding.UTF8.GetBytes(text);
// encrypt the data.
var bOut = new MemoryStream();
var encryptorBldr = provider.CreateEncryptorBuilder(FipsAes.Ctr.WithIV(iv));
var encryptor = encryptorBldr.BuildCipher(bOut);
using (Stream encryptingStream = encryptor.Stream)
{
// Write several copies a byte at a time to prove it works.
writeMultipleCopiesOneByteAtOnce(toBeEncrypted, COPIES, encryptingStream); // Change copies to 60 and it breaks.
}
byte[] cipherText = bOut.ToArray();
// decrypt the resulting cipher text
var decryptorBldr = provider.CreateDecryptorBuilder(FipsAes.Ctr.WithIV(iv));
var decryptor = decryptorBldr.BuildCipher(new MemoryStream(cipherText));
using var decIn = decryptor.Stream;
var bytes = readAllOneByteAtOnce(decIn); // Prove that we can read the decryptor stream a byte at a time.
string result = Encoding.UTF8.GetString(bytes); // Print decrypted text.
Console.WriteLine(result);
}
// Just for test purposes. I wouldn't normally write bytes like this.
static void writeMultipleCopiesOneByteAtOnce(byte[] data, int copies, Stream output)
{
for (int i = 0; i < copies; ++i)
{
foreach (byte b in data)
output.WriteByte(b);
}
}
// Just for test purposes. I wouldn't normally read bytes like this.
static byte[] readAllOneByteAtOnce(Stream stream)
{
using var memStream = new MemoryStream();
while (true)
{
int b = stream.ReadByte();
if (b < 0)
return memStream.ToArray();
memStream.WriteByte((byte)b);
}
}
}
}
I'm a newbie in cryptography and to learn it I tried to encrypt/decrypt with AES in C#.
Sadly I realized, that it isn't as easy as I thought.
So I was looking for a simpler solution.
Later I found a couple of code snippets including some explanation.
I copied the code and tried to implement it into a small application.
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
namespace aes
{
class Program
{
public static string passwd = null;
public static string content = null;
public static string encryptedcontent = null;
public static byte[] IV = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 };
public static int BlockSize = 128;
static void Encrypt()
{
if (passwd == "") return;
//Content to Byte Array
byte[] bytes = Encoding.Unicode.GetBytes(content);
//Encrypt
//Init AES
SymmetricAlgorithm crypt = Aes.Create();
//Init md5 hash
HashAlgorithm hash = MD5.Create();
//AES blocksize (AES 192 etc.) (min 128)
crypt.BlockSize = BlockSize;
//Generating Key
crypt.Key = hash.ComputeHash(Encoding.Unicode.GetBytes(passwd));
//Initialize Vectors
crypt.IV = IV;
//CryptoStram is used for encryption
//The required Encryptor is based on the algorithm above
//Cryptostream sends data of the encrypted byte array to Memorystream
//The memory stream is then converted into a Base64 string and made readable
using (MemoryStream memoryStream = new MemoryStream())
{
using (CryptoStream cryptoStream =
new CryptoStream(memoryStream, crypt.CreateEncryptor(), CryptoStreamMode.Write))
{
cryptoStream.Write(bytes, 0, bytes.Length);
}
encryptedcontent = Convert.ToBase64String(memoryStream.ToArray());
}
}
static void Main(string[] args)
{
//Set Password
Console.WriteLine("Passwort angeben");
Console.Write("> ");
passwd = Console.ReadLine();
//Set content to encrypt (String)
Console.WriteLine("Zu verschlüsselner Text angeben");
Console.Write("> ");
content = Console.ReadLine();
Encrypt();
Console.WriteLine(encryptedcontent);
Console.ReadLine();
}
}
}
Subsequently I wanted to try the programm with some testdata.
I actually got a seemingly encrypted string.
PW: supersecretpassword Content: I like to keep my secrets Result: SEEc1sLMIyfVFsoHPFRIcl437+yjUC5uFMgco3iO+oWSgJWQOwKhoDhUbFJREeqiIvaY2DBR+Ih4OJeGAc6JZQ==
I tried to use some online tools to decrypt and check my result.
Sadly most of the Webtools were not able to decrypt my result.
And if I encrypt the sentence I like to keep my secrets with that online tools I get results like:
7IWuebm0T8HdrGdtkBjt5zgjbdEqYfidNZVvfgtOjH4=
My result SEEc1sLMIyfVFsoHPFRIcl437+yjUC5uFMgco3iO+oWSgJWQOwKhoDhUbFJREeqiIvaY2DBR+Ih4OJeGAc6JZQ==
As you can see, the two results are different.
Unfortunately I have no idea why this could be the case.
Thanks for you help
Jonas
P.S Somehow I deleted some of rows written in this question. I hope the new words can clarify what my problem is.
You don't say what online tools did, or did not, succeed in replicating your results, so this is a general answer, instead of specific.
//AES blocksize (AES 192 etc.) (min 128)
crypt.BlockSize = BlockSize;
The BlockSize of AES is 128. Always (contrast with the original algorithm, Rijndael, which allows the BlockSize to change).
AES-128/AES-192/AES-256 are about the KeySize, not the BlockSize.
crypt.Key = hash.ComputeHash(Encoding.Unicode.GetBytes(passwd));
You're using MD5(UTF16(password)) as your Key Deriviation Function (KDF). Maybe you can find an online sample that is using this, but they're more likely to be using MD5(UTF8(password)) (which would come from Encoding.UTF8, vs Encoding.Unicode). A better answer would be to use a proper password-based Key Derivation Function, like PBKDF2 (which is called Rfc2898DeriveBytes in .NET for... reasons).
[When I encrypt I like to keep my secrets I get an answer that is twice as long as online tools.]
You're encrypting the UTF-16 representation of that string. The string is comprised of 25 Unicode codepoint values, all from the US-ASCII range. Therefore the UTF-16 representation is just the codepoint length * 2 (50 bytes).
50 bytes breaks down into 3 16-byte (128-bit) blocks, plus 2 bytes left over. Add padding, that becomes 4 blocks of AES-CBC-PKCS#7 output (64 bytes). 64 bytes converts to Base64 as 21 full values (of 3 bytes -> 4 chars) with 1 byte remaining, so the Base64 value ends in 2 = padding characters with a total length of 88 characters. This matches your description, hooray :).
If, on the other hand, you used the UTF-8 encoding, you'd have 25 bytes into encryption, which becomes 2 blocks of output (32 bytes) which turns into 10 full base64 conversions with 2 bytes remaining, so one = at a total of 44 characters... which looks a lot like what the online tools are using.
You also should produce a new IV for every time you encrypt with the same key. The IV isn't a key, but changing the IV causes the same secret input to get encrypted differently, so someone who can see your encrypted data can't tell that you sent the same message that you just sent. (At least, that's the purpose in CBC block mode, in other block modes it has sometimes more important purposes). The IV can be transmitted with the message... in fact it should be, unless you have some other way of both sides agreeing (without hard-coding it).
And, of course, you should dispose all of your disposable objects. Changing your encoding to UTF-8, but not changing your KDF, would better be
private static string Encrypt(string content, string password)
{
byte[] bytes = Encoding.UTF8.GetBytes(content);
using (SymmetricAlgorithm crypt = Aes.Create())
using (HashAlgorithm hash = MD5.Create())
using (MemoryStream memoryStream = new MemoryStream())
{
crypt.Key = hash.ComputeHash(Encoding.UTF8.GetBytes(password));
// This is really only needed before you call CreateEncryptor the second time,
// since it starts out random. But it's here just to show it exists.
crypt.GenerateIV();
using (CryptoStream cryptoStream = new CryptoStream(
memoryStream, crypt.CreateEncryptor(), CryptoStreamMode.Write))
{
cryptoStream.Write(bytes, 0, bytes.Length);
}
string base64IV = Convert.ToBase64String(crypt.IV);
string base64Ciphertext = Convert.ToBase64String(memoryStream.ToArray());
return base64IV + "!" + base64Ciphertext;
}
}
Some issues that I see is a self defined IV and odd blocksize, edit: and you probably have the wrong value for the password in mind when comparing to online tools where you do have to fill in the password as calculated by the ComputeHash function.
Check out this simple MSDN Example
I have a working implementation of TripleDESCng (tested against some test vectors), but the following happens:
When I encrypt plain text This is a sample message (24 bytes, thus for this it would be 3 blocks) (hex for it is 5468697320697320612073616D706C65206D657373616765) with an example key, I get E81F113DD7C5D965E082F3D42EC1E2CA39BCDBCCBC0A2BD9. However, when I decrypt this with the same example key, I get 5468697320697320612073616D706C650000000000000000, which, when converted back to ASCII, is:
This is a sample.
Any reason other than my code why this would behave this way? To encrypt and decrypt, I use 24 byte keys (ECB mode).
EDIT:
using (var tripleDES = new TripleDESCryptoServiceProvider())
{
byte[] data = ASCIIEncoding.ASCII.GetBytes("This is a sample message");
Console.WriteLine(BitConverter.ToString(data));
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
encryptor.TransformBlock(data, 0, data.Length, result, 0);
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[result.Length];
decryptor.TransformBlock(result, 0, result.Length, result2, 0);
Console.WriteLine(BitConverter.ToString(result2));
}
Console.ReadLine();
With almost all modes1, you should make sure that the final part of your data is pushed through TransformFinalBlock rather than TransformBlock2, to make sure it knows that no more data is coming and to ensure final blocks are flushed/written.
It's bad form, in general, to assume the output size is going to match the input size.
the mode is not a problem, IV is set to 0s either way
Yes, that'll mean that the first block was not affected by your choice of Mode. But all subsequent blocks will be, because they will use the chaining mode and the previous block, not the IV. So if you want ECB (you shouldn't3) you need to explicitly set that mode.
1Your code is using CBC, not EBC as you claim in your narrative. CBC is the default mode for .NET encryption classes.
2And when using this second method, pay attention to it's return value, as mjwills commented.
3You've picked an outdated crypto algorithm, paired it with an outdated mode of operation, and your words I've quoted above mean that you don't understand modes. Added together, I would suggest that you're not well placed to be writing code that uses crypto currently. The .NET classes can make it seem easy to write crypto code but you still have to understand how to make good choices in using them. Better to spend more time on researching these things before writing code.
I think that your problem is in the method of the encryptor / decryptor that you are using: the TransformBlock method is conceived to transform a block when you will be encrypting multiple blocks.
That is not the case in your code, where you want to transform a single block, and therefore you should be using the TransformFinalBlock method instead. BTW I took the liberty of making your sample buildable.
using System;
using System.Text;
namespace Tests
{
class Program
{
static void Main(string[] args)
{
System.Security.Cryptography.TripleDESCryptoServiceProvider tripleDES = new System.Security.Cryptography.TripleDESCryptoServiceProvider();
byte[] data = Encoding.UTF8.GetBytes("This is a sample message");
byte[] key = Encoding.UTF8.GetBytes("NOSTROMOHASSOMEGODPOWERS");
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
result = encryptor.TransformFinalBlock(data, 0, data.Length);
string res = BitConverter.ToString(result).Replace("-","");
Console.WriteLine(BitConverter.ToString(result).Replace("-",""));
byte[] data2 = result;
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[data2.Length];
result2 = decryptor.TransformFinalBlock(data2, 0, data2.Length);
Console.WriteLine(Encoding.UTF8.GetString(result2));
}
}
}
I have a working implementation of TripleDESCng (tested against some test vectors), but the following happens:
When I encrypt plain text This is a sample message (24 bytes, thus for this it would be 3 blocks) (hex for it is 5468697320697320612073616D706C65206D657373616765) with an example key, I get E81F113DD7C5D965E082F3D42EC1E2CA39BCDBCCBC0A2BD9. However, when I decrypt this with the same example key, I get 5468697320697320612073616D706C650000000000000000, which, when converted back to ASCII, is:
This is a sample.
Any reason other than my code why this would behave this way? To encrypt and decrypt, I use 24 byte keys (ECB mode).
EDIT:
using (var tripleDES = new TripleDESCryptoServiceProvider())
{
byte[] data = ASCIIEncoding.ASCII.GetBytes("This is a sample message");
Console.WriteLine(BitConverter.ToString(data));
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
encryptor.TransformBlock(data, 0, data.Length, result, 0);
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[result.Length];
decryptor.TransformBlock(result, 0, result.Length, result2, 0);
Console.WriteLine(BitConverter.ToString(result2));
}
Console.ReadLine();
With almost all modes1, you should make sure that the final part of your data is pushed through TransformFinalBlock rather than TransformBlock2, to make sure it knows that no more data is coming and to ensure final blocks are flushed/written.
It's bad form, in general, to assume the output size is going to match the input size.
the mode is not a problem, IV is set to 0s either way
Yes, that'll mean that the first block was not affected by your choice of Mode. But all subsequent blocks will be, because they will use the chaining mode and the previous block, not the IV. So if you want ECB (you shouldn't3) you need to explicitly set that mode.
1Your code is using CBC, not EBC as you claim in your narrative. CBC is the default mode for .NET encryption classes.
2And when using this second method, pay attention to it's return value, as mjwills commented.
3You've picked an outdated crypto algorithm, paired it with an outdated mode of operation, and your words I've quoted above mean that you don't understand modes. Added together, I would suggest that you're not well placed to be writing code that uses crypto currently. The .NET classes can make it seem easy to write crypto code but you still have to understand how to make good choices in using them. Better to spend more time on researching these things before writing code.
I think that your problem is in the method of the encryptor / decryptor that you are using: the TransformBlock method is conceived to transform a block when you will be encrypting multiple blocks.
That is not the case in your code, where you want to transform a single block, and therefore you should be using the TransformFinalBlock method instead. BTW I took the liberty of making your sample buildable.
using System;
using System.Text;
namespace Tests
{
class Program
{
static void Main(string[] args)
{
System.Security.Cryptography.TripleDESCryptoServiceProvider tripleDES = new System.Security.Cryptography.TripleDESCryptoServiceProvider();
byte[] data = Encoding.UTF8.GetBytes("This is a sample message");
byte[] key = Encoding.UTF8.GetBytes("NOSTROMOHASSOMEGODPOWERS");
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var encryptor = tripleDES.CreateEncryptor();
byte[] result = new byte[data.Length];
result = encryptor.TransformFinalBlock(data, 0, data.Length);
string res = BitConverter.ToString(result).Replace("-","");
Console.WriteLine(BitConverter.ToString(result).Replace("-",""));
byte[] data2 = result;
tripleDES.Key = key;
tripleDES.IV = new byte[tripleDES.BlockSize / 8];
var decryptor = tripleDES.CreateDecryptor();
byte[] result2 = new byte[data2.Length];
result2 = decryptor.TransformFinalBlock(data2, 0, data2.Length);
Console.WriteLine(Encoding.UTF8.GetString(result2));
}
}
}
Can someone provide some light on how to do this? I can do this for regular text or byte array, but not sure how to approach for a pdf. do i stuff the pdf into a byte array first?
Use File.ReadAllBytes to load the PDF file, and then encode the byte array as normal using Convert.ToBase64String(bytes).
Byte[] fileBytes = File.ReadAllBytes(#"TestData\example.pdf");
var content = Convert.ToBase64String(fileBytes);
There is a way that you can do this in chunks so that you don't have to burn a ton of memory all at once.
.Net includes an encoder that can do the chunking, but it's in kind of a weird place. They put it in the System.Security.Cryptography namespace.
I have tested the example code below, and I get identical output using either my method or Andrew's method above.
Here's how it works: You fire up a class called a CryptoStream. This is kind of an adapter that plugs into another stream. You plug a class called CryptoTransform into the CryptoStream (which in turn is attached to your file/memory/network stream) and it performs data transformations on the data while it's being read from or written to the stream.
Normally, the transformation is encryption/decryption, but .net includes ToBase64 and FromBase64 transformations as well, so we won't be encrypting, just encoding.
Here's the code. I included a (maybe poorly named) implementation of Andrew's suggestion so that you can compare the output.
class Base64Encoder
{
public void Encode(string inFileName, string outFileName)
{
System.Security.Cryptography.ICryptoTransform transform = new System.Security.Cryptography.ToBase64Transform();
using(System.IO.FileStream inFile = System.IO.File.OpenRead(inFileName),
outFile = System.IO.File.Create(outFileName))
using (System.Security.Cryptography.CryptoStream cryptStream = new System.Security.Cryptography.CryptoStream(outFile, transform, System.Security.Cryptography.CryptoStreamMode.Write))
{
// I'm going to use a 4k buffer, tune this as needed
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = inFile.Read(buffer, 0, buffer.Length)) > 0)
cryptStream.Write(buffer, 0, bytesRead);
cryptStream.FlushFinalBlock();
}
}
public void Decode(string inFileName, string outFileName)
{
System.Security.Cryptography.ICryptoTransform transform = new System.Security.Cryptography.FromBase64Transform();
using (System.IO.FileStream inFile = System.IO.File.OpenRead(inFileName),
outFile = System.IO.File.Create(outFileName))
using (System.Security.Cryptography.CryptoStream cryptStream = new System.Security.Cryptography.CryptoStream(inFile, transform, System.Security.Cryptography.CryptoStreamMode.Read))
{
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = cryptStream.Read(buffer, 0, buffer.Length)) > 0)
outFile.Write(buffer, 0, bytesRead);
outFile.Flush();
}
}
// this version of Encode pulls everything into memory at once
// you can compare the output of my Encode method above to the output of this one
// the output should be identical, but the crytostream version
// will use way less memory on a large file than this version.
public void MemoryEncode(string inFileName, string outFileName)
{
byte[] bytes = System.IO.File.ReadAllBytes(inFileName);
System.IO.File.WriteAllText(outFileName, System.Convert.ToBase64String(bytes));
}
}
I am also playing around with where I attach the CryptoStream. In the Encode method,I am attaching it to the output (writing) stream, so when I instance the CryptoStream, I use its Write() method.
When I read, I'm attaching it to the input (reading) stream, so I use the read method on the CryptoStream. It doesn't really matter which stream I attach it to. I just have to pass the appropriate Read or Write enumeration member to the CryptoStream's constructor.