I am currently trying to serialize a List, it serializes (I think fine), but when it deserialize,
Sorry for the amount of code, but I am really stuck and have no idea why this is happening, i also tried to changed the struct into a class and no help.
THANKS.
i get the following error UPDATED
There was an error deserializing the object of type There was an error deserializing the object of type
`System.Collections.Generic.List`1[[A.B.C.DataValues, A.V, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. Unexpected end of file. Following elements are not closed: Time, DataValues, ArrayOfDataValues.`
I am serializing like this UPDATED
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress);
serializer.WriteObject(w, values);
}
_serializedData = stream.ToArray();
}
}
I am deserializing like this UPDATED
public List<DataValues> DeserializeDataValue()
{
if (SerializedData == null || SerializedData.Length == 0)
{
return new List<DataValues> ();
}
else
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream(SerializedData))
{
using (GZipStream decompress = new GZipStream(stream, CompressionMode.Decompress))
{
XmlDictionaryReader r = XmlDictionaryReader.CreateBinaryReader(decompress, XmlDictionaryReaderQuotas.Max);
return serializer.ReadObject(r, true) as List<DataValues>;
}
}
}
}
Properties
private byte[] _serializedData;
[DataMember]
[Browsable(false)]
public byte[] SerializedData
{
get { return _serializedData; }
set { _serializedData = value; }
}
helper Methods
public static byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16 * 1024];
input.Position = 0;
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
Struct
[DataContract(Name = "DataValues", Namespace = "A.B.C")]
public struct DataValues
{
[DataMember]
public DateTime Time { get; set; }
[DataMember]
public Single Value { get; set; }
public DataValues(DateTime dateTime, Single value)
{
Time = dateTime;
Value = value;
}
}
It’s because you are not serialising the object(s) completely. You need to close the stream(s) after writing, especially when using gzip. Recommended practice is to use using:
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress);
serializer.WriteObject(w, values);
}
_serializedData = stream.ToArray();
}
}
Sorry to be late to this question.
The problem with the initial approach was simply that you weren't flushing (read: disposing) the XmlDictionaryWriter.
This should work (note the 2nd using clause):
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
using (XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress))
{
serializer.WriteObject(w, values);
}
Hope this helps someone.
I can get the sample to work by removing the XmlDictionaryReader and instead directly feeding the input/output stream into the DataContractSerializer. It may be a defect in the XmlDictionaryReader for large compressed collections but I'm not sure.
Hope this helps:
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
serializer.WriteObject(compress , values);
}
_serializedData = stream.ToArray();
}
}
public List<DataValues> DeserializeDataValue()
{
if (SerializedData == null || SerializedData.Length == 0)
{
return new List<DataValues> ();
}
else
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream(SerializedData))
{
using (GZipStream decompress = new GZipStream(stream, CompressionMode.Decompress))
{
return serializer.ReadObject(decompress , true) as List<DataValues>;
}
}
}
}
I ran exactly into the same problem and I finally found the solution : the XmlDictionaryWriter needs to be disposed/closed before the Stream you are writing into is itself closed. I discovered that thanks to the thorough example found at http://www.albahari.com/nutshell/ch15.aspx whiche are more complete than the MSDN ones.
In your sample code, that would be :
using (XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress))
{
serializer.WriteObject(w, values);
}
On my own example, using the XmlDictionaryWriter instead of the plain and by default Xml writer only gave me a ~25% decrease in file size but a factor 3 when reading back the object.
Related
I am newbie in .net. I am doing compression and decompression string in C#. There is a XML and I am converting in string and after that I am doing compression and decompression.There is no compilation error in my code except when I decompression my code and return my string, its returning only half of the XML.
Below is my code, please correct me where I am wrong.
Code:
class Program
{
public static string Zip(string value)
{
//Transform string into byte[]
byte[] byteArray = new byte[value.Length];
int indexBA = 0;
foreach (char item in value.ToCharArray())
{
byteArray[indexBA++] = (byte)item;
}
//Prepare for compress
System.IO.MemoryStream ms = new System.IO.MemoryStream();
System.IO.Compression.GZipStream sw = new System.IO.Compression.GZipStream(ms, System.IO.Compression.CompressionMode.Compress);
//Compress
sw.Write(byteArray, 0, byteArray.Length);
//Close, DO NOT FLUSH cause bytes will go missing...
sw.Close();
//Transform byte[] zip data to string
byteArray = ms.ToArray();
System.Text.StringBuilder sB = new System.Text.StringBuilder(byteArray.Length);
foreach (byte item in byteArray)
{
sB.Append((char)item);
}
ms.Close();
sw.Dispose();
ms.Dispose();
return sB.ToString();
}
public static string UnZip(string value)
{
//Transform string into byte[]
byte[] byteArray = new byte[value.Length];
int indexBA = 0;
foreach (char item in value.ToCharArray())
{
byteArray[indexBA++] = (byte)item;
}
//Prepare for decompress
System.IO.MemoryStream ms = new System.IO.MemoryStream(byteArray);
System.IO.Compression.GZipStream sr = new System.IO.Compression.GZipStream(ms,
System.IO.Compression.CompressionMode.Decompress);
//Reset variable to collect uncompressed result
byteArray = new byte[byteArray.Length];
//Decompress
int rByte = sr.Read(byteArray, 0, byteArray.Length);
//Transform byte[] unzip data to string
System.Text.StringBuilder sB = new System.Text.StringBuilder(rByte);
//Read the number of bytes GZipStream red and do not a for each bytes in
//resultByteArray;
for (int i = 0; i < rByte; i++)
{
sB.Append((char)byteArray[i]);
}
sr.Close();
ms.Close();
sr.Dispose();
ms.Dispose();
return sB.ToString();
}
static void Main(string[] args)
{
XDocument doc = XDocument.Load(#"D:\RSP.xml");
string val = doc.ToString(SaveOptions.DisableFormatting);
val = Zip(val);
val = UnZip(val);
}
}
My XML size is 63KB.
The code to compress/decompress a string
public static void CopyTo(Stream src, Stream dest) {
byte[] bytes = new byte[4096];
int cnt;
while ((cnt = src.Read(bytes, 0, bytes.Length)) != 0) {
dest.Write(bytes, 0, cnt);
}
}
public static byte[] Zip(string str) {
var bytes = Encoding.UTF8.GetBytes(str);
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream()) {
using (var gs = new GZipStream(mso, CompressionMode.Compress)) {
//msi.CopyTo(gs);
CopyTo(msi, gs);
}
return mso.ToArray();
}
}
public static string Unzip(byte[] bytes) {
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream()) {
using (var gs = new GZipStream(msi, CompressionMode.Decompress)) {
//gs.CopyTo(mso);
CopyTo(gs, mso);
}
return Encoding.UTF8.GetString(mso.ToArray());
}
}
static void Main(string[] args) {
byte[] r1 = Zip("StringStringStringStringStringStringStringStringStringStringStringStringStringString");
string r2 = Unzip(r1);
}
Remember that Zip returns a byte[], while Unzip returns a string. If you want a string from Zip you can Base64 encode it (for example by using Convert.ToBase64String(r1)) (the result of Zip is VERY binary! It isn't something you can print to the screen or write directly in an XML)
The version suggested is for .NET 2.0, for .NET 4.0 use the MemoryStream.CopyTo.
IMPORTANT: The compressed contents cannot be written to the output stream until the GZipStream knows that it has all of the input (i.e., to effectively compress it needs all of the data). You need to make sure that you Dispose() of the GZipStream before inspecting the output stream (e.g., mso.ToArray()). This is done with the using() { } block above. Note that the GZipStream is the innermost block and the contents are accessed outside of it. The same goes for decompressing: Dispose() of the GZipStream before attempting to access the data.
according to
this snippet
i use this code and it's working fine:
using System;
using System.IO;
using System.IO.Compression;
using System.Text;
namespace CompressString
{
internal static class StringCompressor
{
/// <summary>
/// Compresses the string.
/// </summary>
/// <param name="text">The text.</param>
/// <returns></returns>
public static string CompressString(string text)
{
byte[] buffer = Encoding.UTF8.GetBytes(text);
var memoryStream = new MemoryStream();
using (var gZipStream = new GZipStream(memoryStream, CompressionMode.Compress, true))
{
gZipStream.Write(buffer, 0, buffer.Length);
}
memoryStream.Position = 0;
var compressedData = new byte[memoryStream.Length];
memoryStream.Read(compressedData, 0, compressedData.Length);
var gZipBuffer = new byte[compressedData.Length + 4];
Buffer.BlockCopy(compressedData, 0, gZipBuffer, 4, compressedData.Length);
Buffer.BlockCopy(BitConverter.GetBytes(buffer.Length), 0, gZipBuffer, 0, 4);
return Convert.ToBase64String(gZipBuffer);
}
/// <summary>
/// Decompresses the string.
/// </summary>
/// <param name="compressedText">The compressed text.</param>
/// <returns></returns>
public static string DecompressString(string compressedText)
{
byte[] gZipBuffer = Convert.FromBase64String(compressedText);
using (var memoryStream = new MemoryStream())
{
int dataLength = BitConverter.ToInt32(gZipBuffer, 0);
memoryStream.Write(gZipBuffer, 4, gZipBuffer.Length - 4);
var buffer = new byte[dataLength];
memoryStream.Position = 0;
using (var gZipStream = new GZipStream(memoryStream, CompressionMode.Decompress))
{
gZipStream.Read(buffer, 0, buffer.Length);
}
return Encoding.UTF8.GetString(buffer);
}
}
}
}
With the advent of .NET 4.0 (and higher) with the Stream.CopyTo() methods, I thought I would post an updated approach.
I also think the below version is useful as a clear example of a self-contained class for compressing regular strings to Base64 encoded strings, and vice versa:
public static class StringCompression
{
/// <summary>
/// Compresses a string and returns a deflate compressed, Base64 encoded string.
/// </summary>
/// <param name="uncompressedString">String to compress</param>
public static string Compress(string uncompressedString)
{
byte[] compressedBytes;
using (var uncompressedStream = new MemoryStream(Encoding.UTF8.GetBytes(uncompressedString)))
{
using (var compressedStream = new MemoryStream())
{
// setting the leaveOpen parameter to true to ensure that compressedStream will not be closed when compressorStream is disposed
// this allows compressorStream to close and flush its buffers to compressedStream and guarantees that compressedStream.ToArray() can be called afterward
// although MSDN documentation states that ToArray() can be called on a closed MemoryStream, I don't want to rely on that very odd behavior should it ever change
using (var compressorStream = new DeflateStream(compressedStream, CompressionLevel.Fastest, true))
{
uncompressedStream.CopyTo(compressorStream);
}
// call compressedStream.ToArray() after the enclosing DeflateStream has closed and flushed its buffer to compressedStream
compressedBytes = compressedStream.ToArray();
}
}
return Convert.ToBase64String(compressedBytes);
}
/// <summary>
/// Decompresses a deflate compressed, Base64 encoded string and returns an uncompressed string.
/// </summary>
/// <param name="compressedString">String to decompress.</param>
public static string Decompress(string compressedString)
{
byte[] decompressedBytes;
var compressedStream = new MemoryStream(Convert.FromBase64String(compressedString));
using (var decompressorStream = new DeflateStream(compressedStream, CompressionMode.Decompress))
{
using (var decompressedStream = new MemoryStream())
{
decompressorStream.CopyTo(decompressedStream);
decompressedBytes = decompressedStream.ToArray();
}
}
return Encoding.UTF8.GetString(decompressedBytes);
}
}
Here’s another approach using the extension methods technique to extend the String class to add string compression and decompression. You can drop the class below into an existing project and then use thusly:
var uncompressedString = "Hello World!";
var compressedString = uncompressedString.Compress();
and
var decompressedString = compressedString.Decompress();
To wit:
public static class Extensions
{
/// <summary>
/// Compresses a string and returns a deflate compressed, Base64 encoded string.
/// </summary>
/// <param name="uncompressedString">String to compress</param>
public static string Compress(this string uncompressedString)
{
byte[] compressedBytes;
using (var uncompressedStream = new MemoryStream(Encoding.UTF8.GetBytes(uncompressedString)))
{
using (var compressedStream = new MemoryStream())
{
// setting the leaveOpen parameter to true to ensure that compressedStream will not be closed when compressorStream is disposed
// this allows compressorStream to close and flush its buffers to compressedStream and guarantees that compressedStream.ToArray() can be called afterward
// although MSDN documentation states that ToArray() can be called on a closed MemoryStream, I don't want to rely on that very odd behavior should it ever change
using (var compressorStream = new DeflateStream(compressedStream, CompressionLevel.Fastest, true))
{
uncompressedStream.CopyTo(compressorStream);
}
// call compressedStream.ToArray() after the enclosing DeflateStream has closed and flushed its buffer to compressedStream
compressedBytes = compressedStream.ToArray();
}
}
return Convert.ToBase64String(compressedBytes);
}
/// <summary>
/// Decompresses a deflate compressed, Base64 encoded string and returns an uncompressed string.
/// </summary>
/// <param name="compressedString">String to decompress.</param>
public static string Decompress(this string compressedString)
{
byte[] decompressedBytes;
var compressedStream = new MemoryStream(Convert.FromBase64String(compressedString));
using (var decompressorStream = new DeflateStream(compressedStream, CompressionMode.Decompress))
{
using (var decompressedStream = new MemoryStream())
{
decompressorStream.CopyTo(decompressedStream);
decompressedBytes = decompressedStream.ToArray();
}
}
return Encoding.UTF8.GetString(decompressedBytes);
}
}
I like #fubo's answer the best but I think this is much more elegant.
This method is more compatible because it doesn't manually store the length up front.
Also I've exposed extensions to support compression for string to string, byte[] to byte[], and Stream to Stream.
public static class ZipExtensions
{
public static string CompressToBase64(this string data)
{
return Convert.ToBase64String(Encoding.UTF8.GetBytes(data).Compress());
}
public static string DecompressFromBase64(this string data)
{
return Encoding.UTF8.GetString(Convert.FromBase64String(data).Decompress());
}
public static byte[] Compress(this byte[] data)
{
using (var sourceStream = new MemoryStream(data))
using (var destinationStream = new MemoryStream())
{
sourceStream.CompressTo(destinationStream);
return destinationStream.ToArray();
}
}
public static byte[] Decompress(this byte[] data)
{
using (var sourceStream = new MemoryStream(data))
using (var destinationStream = new MemoryStream())
{
sourceStream.DecompressTo(destinationStream);
return destinationStream.ToArray();
}
}
public static void CompressTo(this Stream stream, Stream outputStream)
{
using (var gZipStream = new GZipStream(outputStream, CompressionMode.Compress))
{
stream.CopyTo(gZipStream);
gZipStream.Flush();
}
}
public static void DecompressTo(this Stream stream, Stream outputStream)
{
using (var gZipStream = new GZipStream(stream, CompressionMode.Decompress))
{
gZipStream.CopyTo(outputStream);
}
}
}
This is an updated version for .NET 4.5 and newer using async/await and IEnumerables:
public static class CompressionExtensions
{
public static async Task<IEnumerable<byte>> Zip(this object obj)
{
byte[] bytes = obj.Serialize();
using (MemoryStream msi = new MemoryStream(bytes))
using (MemoryStream mso = new MemoryStream())
{
using (var gs = new GZipStream(mso, CompressionMode.Compress))
await msi.CopyToAsync(gs);
return mso.ToArray().AsEnumerable();
}
}
public static async Task<object> Unzip(this byte[] bytes)
{
using (MemoryStream msi = new MemoryStream(bytes))
using (MemoryStream mso = new MemoryStream())
{
using (var gs = new GZipStream(msi, CompressionMode.Decompress))
{
// Sync example:
//gs.CopyTo(mso);
// Async way (take care of using async keyword on the method definition)
await gs.CopyToAsync(mso);
}
return mso.ToArray().Deserialize();
}
}
}
public static class SerializerExtensions
{
public static byte[] Serialize<T>(this T objectToWrite)
{
using (MemoryStream stream = new MemoryStream())
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
binaryFormatter.Serialize(stream, objectToWrite);
return stream.GetBuffer();
}
}
public static async Task<T> _Deserialize<T>(this byte[] arr)
{
using (MemoryStream stream = new MemoryStream())
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
await stream.WriteAsync(arr, 0, arr.Length);
stream.Position = 0;
return (T)binaryFormatter.Deserialize(stream);
}
}
public static async Task<object> Deserialize(this byte[] arr)
{
object obj = await arr._Deserialize<object>();
return obj;
}
}
With this you can serialize everything BinaryFormatter supports, instead only of strings.
Edit:
In case, you need take care of Encoding, you could just use Convert.ToBase64String(byte[])...
Take a look at this answer if you need an example!
For those who still getting The magic number in GZip header is not correct. Make sure you are passing in a GZip stream. ERROR
and if your string was zipped using php you'll need to do something like:
public static string decodeDecompress(string originalReceivedSrc) {
byte[] bytes = Convert.FromBase64String(originalReceivedSrc);
using (var mem = new MemoryStream()) {
//the trick is here
mem.Write(new byte[] { 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00 }, 0, 8);
mem.Write(bytes, 0, bytes.Length);
mem.Position = 0;
using (var gzip = new GZipStream(mem, CompressionMode.Decompress))
using (var reader = new StreamReader(gzip)) {
return reader.ReadToEnd();
}
}
}
We can reduce code complexity by using StreamReader and StreamWriter rather than manually converting strings to byte arrays. Three streams is all you need:
public static byte[] Zip(string uncompressed)
{
byte[] ret;
using (var outputMemory = new MemoryStream())
{
using (var gz = new GZipStream(outputMemory, CompressionLevel.Optimal))
{
using (var sw = new StreamWriter(gz, Encoding.UTF8))
{
sw.Write(uncompressed);
}
}
ret = outputMemory.ToArray();
}
return ret;
}
public static string Unzip(byte[] compressed)
{
string ret = null;
using (var inputMemory = new MemoryStream(compressed))
{
using (var gz = new GZipStream(inputMemory, CompressionMode.Decompress))
{
using (var sr = new StreamReader(gz, Encoding.UTF8))
{
ret = sr.ReadToEnd();
}
}
}
return ret;
}
For .net6 cross platform Compression/Decompression string with C# using SharpZipLib library. Test for ubuntu(18.0.x) and windows.
#region helper
private byte[] Zip(string text)
{
if (text == null)
return null;
byte[] ret;
using (var outputMemory = new MemoryStream())
{
using (var gz = new GZipStream(outputMemory, CompressionLevel.Optimal))
{
using (var sw = new StreamWriter(gz, Encoding.UTF8))
{
sw.Write(text);
}
}
ret = outputMemory.ToArray();
}
return ret;
}
private string Unzip(byte[] bytes)
{
string ret = null;
using (var inputMemory = new MemoryStream(bytes))
{
using (var gz = new GZipStream(inputMemory, CompressionMode.Decompress))
{
using (var sr = new StreamReader(gz, Encoding.UTF8))
{
ret = sr.ReadToEnd();
}
}
}
return ret;
}
#endregion
Here is code which takes almost 3 to 8 sec for serialization depends on object type. I want to store this result into Redis for caching. But this operation is taking too long. Also same for deserialization.
public byte[] SerializeObject(object objectToSerialize)
{
try
{
//If object to serialize is null then return null
if (objectToSerialize == null)
return null;
byte[] result;
//Create memory stream and use it for Serializing object
using (var ms = new MemoryStream())
{
using (var zs = new GZipStream(ms, CompressionMode.Compress, true))
{
var bf = new BinaryFormatter();
bf.Serialize(zs, objectToSerialize);
}
result = ms.ToArray();
}
return result;
}
catch (Exception ex)
{
//Some code
return null;
}
}
Updated: How I have tried serializing a DataTable:
using(var client = m_oRedisClientsManager.GetClient()) {
//Serialize DataTable to byte
byteCachedDatatable = m_oSerializer.SerializeDataTable(oCacheObject);
//add Serialized bytes to redis and update expiration time
client.Set(sCacheKey, byteCachedDatatable, new TimeSpan(0, iExpiryTimeInMins, 0));
}
public byte[] SerializeDataTable(DataTable dataTable) {
if (dataTable == null) return null;
byte[] result;
using(var memoryStream = new MemoryStream()) {
using(var deflateStream = new DeflateStream(memoryStream, CompressionMode.Compress)) {
dataTable.WriteXml(deflateStream, XmlWriteMode.WriteSchema);
deflateStream.Flush();
deflateStream.Close();
result = memoryStream.ToArray();
}
}
return result;
}
Any pointers will be helpful.
I built (based on a CodeProject article) a wrapper class (C#) to use a GZipStream to compress a MemoryStream. It compresses fine but doesn't decompress. I've looked at many other examples that have the same problem, and I feel like I'm following what's said but still am getting nothing when I decompress. Here's the compression and decompression methods:
public static byte[] Compress(byte[] bSource)
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Compress, true))
{
gzip.Write(bSource, 0, bSource.Length);
gzip.Close();
}
return ms.ToArray();
}
}
public static byte[] Decompress(byte[] bSource)
{
try
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Decompress, true))
{
gzip.Read(bSource, 0, bSource.Length);
gzip.Close();
}
return ms.ToArray();
}
}
catch (Exception ex)
{
throw new Exception("Error decompressing byte array", ex);
}
}
Here's an example of how I use it:
string sCompressed = Convert.ToBase64String(CompressionHelper.Compress("Some Text"));
// Other Processes
byte[] bReturned = CompressionHelper.Decompress(Convert.FromBase64String(sCompressed));
// bReturned has no elements after this line is executed
There is a bug in Decompress method.
The code does not read content of bSource. On the contrary, it overrides its content wile reading from empty gzip, created based on empty memory stream.
Basically what your version of code is doing:
//create empty memory
using (MemoryStream ms = new MemoryStream())
//create gzip stream over empty memory stream
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Compress, true))
// write from empty stream to bSource
gzip.Write(bSource, 0, bSource.Length);
The fix could look like this:
public static byte[] Decompress(byte[] bSource)
{
using (var inStream = new MemoryStream(bSource))
using (var gzip = new GZipStream(inStream, CompressionMode.Decompress))
using (var outStream = new MemoryStream())
{
gzip.CopyTo(outStream);
return outStream.ToArray();
}
}
The OP said in an edit, now rolled back:
Thanks to Alex's explanation of what was going wrong, I was able to fix the Decompress method. Unfortunately, I'm using .Net 3.5, so I wasn't able to implement the Stream.CopyTo method he suggested. With his explanation, though, I was able to figure out a solution. I made the appropriate changes to the Decompress method below.
public static byte[] Decompress(byte[] bSource)
{
try
{
using (var instream = new MemoryStream(bSource))
{
using (var gzip = new GZipStream(instream, CompressionMode.Decompress))
{
using (var outstream = new MemoryStream())
{
byte[] buffer = new byte[4096];
while (true)
{
int delta = gzip.Read(buffer, 0, buffer.Length);
if (delta > 0)
outstream.Write(buffer, 0, delta);
if (delta < 4096)
break;
}
return outstream.ToArray();
}
}
}
}
catch (Exception ex)
{
throw new Exception("Error decompressing byte array", ex);
}
}
I'm trying to do custom serialization/deserialization of an object as well as compressing/decompressing the serialized data with DeflateStreams. I originally did this for more complex objects but cut it down to try and figure out the problem, however it just became more puzzling as it is still there. Here is the class to be serialized/deserialized:
[Serializable]
public class RandomObject : ISerializable
{
public String Name { get; set; }
public String SavePath { get; set; }
public RandomObject()
{
}
public RandomObject(String name, String savepath)
{
Name = name;
SavePath = savepath;
}
public RandomObject(SerializationInfo info, StreamingContext context)
: this(info.GetString("name"), info.GetString("savepath"))
{
}
[SecurityPermissionAttribute(SecurityAction.Demand, SerializationFormatter = true)]
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("name", Name);
info.AddValue("savepath", SavePath);
}
}
And here is the code that is supposed to serialize it(which seems to work):
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream ms = new MemoryStream())
{
bf.Serialize(ms, profile);
using (DeflateStream ds = new DeflateStream(ms, CompressionMode.Compress))
{
try
{
using (FileStream fs = File.Create(path))
{
ds.Flush();
Miscellaneous.CopyStream(ds.BaseStream, fs);
fs.Flush();
fs.Close();
}
}
catch (IOException e)
{
MessageBox.Show(e.Message);
success = false;
}
ds.Close();
}
ms.Close();
}
And here is the deserialization:
RandomObject profile = null;
using (FileStream fs = File.OpenRead(path))
{
using (DeflateStream ds = new DeflateStream(fs, CompressionMode.Decompress))
{
BinaryFormatter bf = new BinaryFormatter();
ds.Flush();
using (MemoryStream ms = new MemoryStream())
{
Miscellaneous.CopyStream(ds.BaseStream, ms);
profile = bf.Deserialize(ms) as RandomObject;
profile.SavePath = path;
ms.Close();
}
ds.Close();
}
fs.Close();
}
Now, to the problem. Deserialization throws a SerializationException with the message {"No map for object '201326592'."} I have no idea how to troubleshoot or figure out what exactly is causing the problem. Very basic serialization works when I just run BinaryFormatter's Serialize and Deserialize methods on the same MemoryStream.
I tried removing the DeflateStream stuff from both methods, but it's still the same problem. When I look at the examples at MSDN and other places it looks like I'm doing it just right, and googling for the exception message doesn't give any meaningful results(or perhaps I'm just bad at searching).
PS. As you can see I use Miscellaneous.CopyStream(src, dest) which is a basic stream copier, as I can't get src.CopyTo(dest) to work at all, so any hints on that is welcome as well.
Below is a link to the whole VS2010 project if you would like to look at it more closely:
http://www.diredumplings.com/SerializationTesting.zip
UPDATE:
The_Smallest: I tried using the Compress method you posted on my serialization:
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream stream = new MemoryStream())
{
bf.Serialize(stream, profile);
byte[] array = Compress(stream);
using (MemoryStream ms = new MemoryStream(array))
{
using (FileStream fs = File.Create(path))
{
ms.WriteTo(fs);
fs.Flush();
}
}
}
However, it seems to give me the same problems that I had with srcStream.CopyTo(destStream) earlier, which is that it doesn't seem to get written to the stream. The result is a 0 kb file when I try to save it to disk. Any ideas?
Pieter: I removed the MemoryStream from the deserialization method and it seems have the same functionality as before. However I'm not sure how to implement the serialization the way you suggested. Is this what you had in mind?
BinaryFormatter bf = new BinaryFormatter();
using (FileStream fs = File.Create(path))
{
using (DeflateStream ds = new DeflateStream(fs, CompressionMode.Compress))
{
bf.Serialize(ds, profile);
fs.Flush();
ds.Close();
}
fs.Close();
}
Thanks to both of you!
I dowloaded you example and digged a little in there. See changes for your project below:
Replace LoadFromFile in Loader.cs
private static RandomObject LoadFromFile(string path)
{
try
{
var bf = new BinaryFormatter();
using (var fileStream = File.OpenRead(path))
using (var decompressed = new MemoryStream())
{
using (var deflateStream = new DeflateStream(fileStream, CompressionMode.Decompress))
deflateStream.CopyTo(decompressed);
decompressed.Seek(0, SeekOrigin.Begin);
var profile = (RandomObject)bf.Deserialize(decompressed);
profile.SavePath = path;
return profile;
}
}
catch (IOException e)
{
MessageBox.Show(e.Message);
return null;
}
}
Replace Save in Saver.cs as follows:
public static bool Save(RandomObject profile, String path)
{
try
{
var bf = new BinaryFormatter();
using (var uncompressed = new MemoryStream())
using (var fileStream = File.Create(path))
{
bf.Serialize(uncompressed, profile);
uncompressed.Seek(0, SeekOrigin.Begin);
using (var deflateStream = new DeflateStream(fileStream, CompressionMode.Compress))
uncompressed.CopyTo(deflateStream);
}
return true;
}
catch (IOException e)
{
MessageBox.Show(e.Message);
return false;
}
}
You should serialize into the DeflateStream, not the base (MemoryStream) stream.
For serializing: begin with the File.Create. Then around that stream, create the DeflateStream. Then to the DefaulteStream, serialize your objects.
For deserializing: do not create the MemoryStream and deserialize directly from the DeflateStream.
I believe there is no need for the added MemoryStream. If however you do have problems writing directly to/reading directly from the file streams, just change the serialize routine to write to the DeflateStream instead of the MemoryStream.
That should solve your issues.
There is error in streams logics, while compressing you should write to CompressStream, which writes to MemoryStream, after this you will have result in MemoryStream (not in CompressStream)
Here is example how to compress and decompress bytes
private static byte[] Compress(Stream stream)
{
using (var resultStream = new MemoryStream())
{
using (var gzipStream = new DeflateStream(resultStream, CompressionMode.Compress))
stream.CopyTo(gzipStream);
return resultStream.ToArray();
}
}
private static byte[] Decompress(byte[] bytes)
{
using (var readStream = new MemoryStream(bytes))
using (var resultStream = new MemoryStream())
{
using (var gzipStream = new DeflateStream(readStream, CompressionMode.Decompress))
gzipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
}
I am created a post before " Object to byte not working " .
I fixed problems that users said me , but there is still problem .
Error Message : The constructor to deserialize an object of type 'WindowsFormsApplication1.Form1+Item' was not found.;
void start()
{
Item item = new Item();
item.files.Add(#"test");
byte[] b = ObjectToByteArray(item);
Item k = Desriles(b);
}
[Serializable]
public class Item : ISerializable
{
public Item()
{
files = new List<string>();
Exclude = false;
CleanEmptyFolder = false;
}
public List<string> files;
public string MusicProfileName;
public bool Exclude;
#region ISerializable Members
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("files", files);
info.AddValue("MusicProfileName", MusicProfileName);
info.AddValue("Exclude", Exclude);
}
#endregion
}
public byte[] ObjectToByteArray(object _Object)
{
using (var stream = new MemoryStream())
{
// serialize object
var formatter = new BinaryFormatter();
formatter.Serialize(stream, _Object);
// get a byte array
var bytes = new byte[stream.Length];
using (BinaryReader br = new BinaryReader(stream))
{
bytes = br.ReadBytes(Convert.ToInt32(stream.Length));
}
return bytes;
}
}
public Item Desriles(byte[] items)
{
using (MemoryStream stream = new MemoryStream())
{
stream.SetLength(items.LongLength);
stream.write(items, 0, items.Length);
var formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
object item = formatter.Deserialize(stream); // Here I will get error
return (Item)item;
}
}
The serialization code can't work properly, you forgot to reset the stream back to the beginning. The better mouse trap:
public byte[] ObjectToByteArray(object _Object) {
using (var stream = new MemoryStream()) {
var formatter = new BinaryFormatter();
formatter.Serialize(stream, _Object);
return stream.ToArray();
}
}
The deserialization code can similarly be simplified:
public Item Desriles(byte[] items) {
using (MemoryStream stream = new MemoryStream(items)) {
var formatter = new BinaryFormatter();
return (Item)formatter.Deserialize(stream);
}
}
And you don't need GetObjectData().
In this section:
using (MemoryStream stream = new MemoryStream())
{
stream.SetLength(items.LongLength);
stream.Read(items, 0, items.Length);
[...]
object item = formatter.Deserialize(stream);
it seems you are creating a new, empty memory stream, then attempting to read from it, and Deserialize from it.
Of course it fails. The stream is empty.
abelenky makes a good point, but I don't think your:
public byte[] ObjectToByteArray(object _Object)
works either.
C# Code Snippet - Object to byte array
C# Code Snippet - Byte array to object
Thanks to All :
I found problem : I should base that to ISerializable