I am trying to serialize a Hashtable into a string and then back into a hash table but my code for deserializing keeps failing.
To Deserialize:
byte[] bytes = System.Text.Encoding.Unicode.GetBytes(hash.value);
System.IO.MemoryStream ms = new System.IO.MemoryStream();
ms.Write (bytes,0,bytes.Length);
ms.Seek(0,System.IO.SeekOrigin.Begin);
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
Hashtable h = (Hashtable)bf.Deserialize(ms);
To Serialize:
MemoryStream ms = new MemoryStream();
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
bf.Serialize(ms, hash);
byte[] bt = ms.ToArray();
this.value=System.Text.Encoding.Unicode.GetString(bt);
This is the error I am getting with the above code:
EndOfStreamException: Failed to read past end of stream.
But if anyone has a more efficient way to do this I am all ears.
Unicode encoding is the problem. Serialized bytes consist of random bytes, including illegel bytes for the unicode encoding. Use System.Convert.ToBase64String and System.Convert.FromBase64String instead of Encoding.Unicode.GetString and Encoding.Unicode.GetBytes.
But why are you using a string as a medium? An array of bytes is enough to save and load data.
Related
From what I understand, the Formatter converts a serializable object into a stream of bytes and the Stream (e.g. FileStream) does the actual writing of those bytes into a file. Example code:
public static void SaveData(MySerializableData data)
{
BinaryFormatter formatter = new BinaryFormatter();
FileStream stream = new FileStream(SavePath, FileMode.Create);
formatter.Serialize(stream, data);
stream.Close();
}
However at other times, I'm also seeing this type of code:
void Save()
{
string data = "Value 1";
StreamWriter writer = new StreamWriter("MySaveFile.txt", true);
writer.Write(data);
}
Why is it that in the second case, we abandon the 2-step process of saving? Why do we sometimes use only StreamWriter, but at other times use both the formatter and a stream object?
Thank you!
BinaryFormatter serializes a class into a byte array and writes it to a stream. It's useful when you want to save/load classes with it's data. Serializtion stores metadata about the class graph that is storing.
StreamWriter is a stream that has specific functions to write a string into a file.
Consider this example:
MemoryStream mstr = new MemoryStream();
string datastr = "hello!";
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(mstr, datastr);
mstr.Seek(0, SeekOrigin.Begin);
string resultString = Encoding.ASCII.GetString(mstr.ToArray());
If you inspect resultString you will find that it contains something like this:
"\0\u0001\0\0\0????\u0001\0\0\0\0\0\0\0\u0006\u0001\0\0\0\u0006hello!\v"
Well, that's not what you would want to have in a text file, right? As you see serialization is not intended to store raw data but class instances.
Now check this:
MemoryStream mstr = new MemoryStream();
StreamWriter sw = new StreamWriter(mstr);
string datastr = "hello!";
sw.Write(datastr);
mstr.Seek(0, SeekOrigin.Begin);
sw.Close();
string resultString = Encoding.ASCII.GetString(mstr.ToArray());
If you now inspect resultString it will contain:
"hello!"
As you see it's very different, that's what you would expect in a text file.
You can also store raw binary data with a stream:
byte[] data = new byte[]{ 1,2,3,4 };
var fs = File.Create("out.dat"); //this creates a new file and creates a filestream
fs.Write(data, 0, data.Length);
fs.Close();
If you now inspect the file with a binary editor you will see it contains:
0x01, 0x02, 0x03, 0x04
There are many types of streams with different purposes (per example, the MemoryStream that I used in the examples, a binary stream that stores it's data into an array in memory) and a ton of classes that use streams for many things, in this case you have mixed the concepts of serialization and data storage using streams, those are two different things.
I am using AWS KMS Encrypt API to encrypt a text. The Encrypt API returns a EncryptResponse object that has a MemoryStream attribute which is the encrypted value.
I am using the AWS .Net SDK and trying to save this value to DynamoDB.
How can I save this value? I tried converting this to a String but while the conversion saving DynamoDB and retrieval gave me back the MemoryStream object, the resultant cipher text from the recreated MemoryStream object cannot be decrypted with an error "Invalid ciphertext."
Here is the code used to save and retrieve the string
// convert stream to string
StreamReader reader = new StreamReader(ciphertextBlob);
string text = reader.ReadToEnd();
// convert string to stream
byte[] byteArray = Encoding.UTF8.GetBytes(text);
MemoryStream stream = new MemoryStream(byteArray);
When retrieving the stream it was noticed that the ciphertext was different for the object before saving and that during retrieval.
The EncryptFunction
The EncryptResponse
This issue was solved by converting the memory stream object into a byte[] and saving it as a String in DynamoDB.
byte[] byteArray = ReadFully(ciphertextBlob);
string AccessKeycipherText = Convert.ToBase64String(byteArray);
The base64 encoding was used to get a String representation of the byte array.
The memory stream object was retrieved using the reverse.
byte[] temp_backToBytes = Convert.FromBase64String(AccessKey);
MemoryStream stream = new MemoryStream(temp_backToBytes);
I am trying to serialize a class object using binary serialization in C#. I have tried and everywhere all I can find that the serialized data goes to a file always in all the examples I have seen.
In my case, I have to store the serialized data in SQL. The following is an example of the method I have created.
//Serializing the List
public void Serialize(Employees emps, String filename)
{
//Create the stream to add object into it.
System.IO.Stream ms = File.OpenWrite(filename);
//Format the object as Binary
BinaryFormatter formatter = new BinaryFormatter();
//It serialize the employee object
formatter.Serialize(ms, emps);
ms.Flush();
ms.Close();
ms.Dispose();
}
How can I get the serialized data directly in a string variable? I don't want to use a file.
Please help.
The easiest way to represent a byte array as a string in C# is with base64 encoding. The below example shows how this would be achieved within your code.
public void Serialize(Employees emps, String filename)
{
//Create the stream to add object into it.
MemoryStream ms = new MemoryStream();
//Format the object as Binary
BinaryFormatter formatter = new BinaryFormatter();
//It serialize the employee object
formatter.Serialize(ms, emps);
// Your employees object serialised and converted to a string.
string encodedObject = Convert.ToBase64String(ms.ToArray());
ms.Close();
}
This creates the string encodedObject. To retrieve the byte array and your serialised object back from the string you will use the below code.
BinaryFormatter bf = new BinaryFormatter();
// Decode the string back to a byte array
byte[] decodedObject = Convert.FromBase64String(encodedObject);
// Create a memory stream and pass in the decoded byte array as the parameter
MemoryStream memoryStream = new MemoryStream(decodedObject);
// Deserialise byte array back to employees object.
Employees employees = bf.Deserialize(memoryStream);
Just use MemoryStream ms = new MemoryStream() instead of your file stream.
You can extract a byte[] for Storage to SQL after serializing by calling ms.ToArray().
And don't forget to put your Stream into a using-Statement, to guarantee correct disposal of the allocated resources.
I am given an byte array of compressed data, which takes up 27878 bytes. It starts with
78-9C-ED-BD-09
so it should be readable with DeflateStream, which it is. The uncompressed data is a XML file. I want to modify some values in this XML and then compress it again, to save it back to its source.
But even without any modifications, just decompressing and compressing again the result differs from the source, which causes the target-applications, which reads this byte array to crash.
For compression and uncompression I used these methods found in Stackoverflow
private static MemoryStream Decompress(byte[] input)
{
var output = new MemoryStream();
using (var compressStream = new MemoryStream(input))
using (var decompressor = new DeflateStream(compressStream, CompressionMode.Decompress))
decompressor.CopyTo(output);
output.Position = 0;
return output;
}
public byte[] Compress(byte[] input)
{
MemoryStream stream = new MemoryStream();
DeflateStream compressionStream =
new DeflateStream(stream, CompressionMode.Compress);
compressionStream.Write(input, 0, input.Length);
compressionStream.Close();
return stream.ToArray();
}
Edit 04/23/19
As it was pointed out be a comment, that has been deleted, the deflate methods are not suited to compress data. Instead, with the DotNetZip library and some work in this library it was possible to create the same data again!
I'm trying to append some data to a stream. This works well with FileStream, but not for MemoryStream due to the fixed buffer size.
The method which writes data to the stream is separated from the method which creates the stream (I've simplified it greatly in the below example). The method which creates the stream is unaware of the length of data to be written to the stream.
public void Foo(){
byte[] existingData = System.Text.Encoding.UTF8.GetBytes("foo");
Stream s1 = new FileStream("someFile.txt", FileMode.Append, FileAccess.Write, FileShare.Read);
s1.Write(existingData, 0, existingData.Length);
Stream s2 = new MemoryStream(existingData, 0, existingData.Length, true);
s2.Seek(0, SeekOrigin.End); //move to end of the stream for appending
WriteUnknownDataToStream(s1);
WriteUnknownDataToStream(s2); // NotSupportedException is thrown as the MemoryStream is not expandable
}
public static void WriteUnknownDataToStream(Stream s)
{
// this is some example data for this SO query - the real data is generated elsewhere and is of a variable, and often large, size.
byte[] newBytesToWrite = System.Text.Encoding.UTF8.GetBytes("bar"); // the length of this is not known before the stream is created.
s.Write(newBytesToWrite, 0, newBytesToWrite.Length);
}
An idea I had was to send an expandable MemoryStream to the function, then append the returned data to the existing data.
public void ModifiedFoo()
{
byte[] existingData = System.Text.Encoding.UTF8.GetBytes("foo");
Stream s2 = new MemoryStream(); // expandable capacity memory stream
WriteUnknownDataToStream(s2);
// append the data which has been written into s2 to the existingData
byte[] buffer = new byte[existingData.Length + s2.Length];
Buffer.BlockCopy(existingData, 0, buffer, 0, existingData.Length);
Stream merger = new MemoryStream(buffer, true);
merger.Seek(existingData.Length, SeekOrigin.Begin);
s2.CopyTo(merger);
}
Any better (more efficient) solutions?
A possible solution is not to limit the capacity of the MemoryStream in the first place.
If you do not know in advance the total number of bytes you will need to write, create a MemoryStream with unspecified capacity and use it for both writes.
byte[] existingData = System.Text.Encoding.UTF8.GetBytes("foo");
MemoryStream ms = new MemoryStream();
ms.Write(existingData, 0, existingData.Length);
WriteUnknownData(ms);
This will no doubt be less performant than initializing a MemoryStream from a byte[], but if you need to continue writing to the stream I believe it is your only option.