I'm trying to do custom serialization/deserialization of an object as well as compressing/decompressing the serialized data with DeflateStreams. I originally did this for more complex objects but cut it down to try and figure out the problem, however it just became more puzzling as it is still there. Here is the class to be serialized/deserialized:
[Serializable]
public class RandomObject : ISerializable
{
public String Name { get; set; }
public String SavePath { get; set; }
public RandomObject()
{
}
public RandomObject(String name, String savepath)
{
Name = name;
SavePath = savepath;
}
public RandomObject(SerializationInfo info, StreamingContext context)
: this(info.GetString("name"), info.GetString("savepath"))
{
}
[SecurityPermissionAttribute(SecurityAction.Demand, SerializationFormatter = true)]
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("name", Name);
info.AddValue("savepath", SavePath);
}
}
And here is the code that is supposed to serialize it(which seems to work):
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream ms = new MemoryStream())
{
bf.Serialize(ms, profile);
using (DeflateStream ds = new DeflateStream(ms, CompressionMode.Compress))
{
try
{
using (FileStream fs = File.Create(path))
{
ds.Flush();
Miscellaneous.CopyStream(ds.BaseStream, fs);
fs.Flush();
fs.Close();
}
}
catch (IOException e)
{
MessageBox.Show(e.Message);
success = false;
}
ds.Close();
}
ms.Close();
}
And here is the deserialization:
RandomObject profile = null;
using (FileStream fs = File.OpenRead(path))
{
using (DeflateStream ds = new DeflateStream(fs, CompressionMode.Decompress))
{
BinaryFormatter bf = new BinaryFormatter();
ds.Flush();
using (MemoryStream ms = new MemoryStream())
{
Miscellaneous.CopyStream(ds.BaseStream, ms);
profile = bf.Deserialize(ms) as RandomObject;
profile.SavePath = path;
ms.Close();
}
ds.Close();
}
fs.Close();
}
Now, to the problem. Deserialization throws a SerializationException with the message {"No map for object '201326592'."} I have no idea how to troubleshoot or figure out what exactly is causing the problem. Very basic serialization works when I just run BinaryFormatter's Serialize and Deserialize methods on the same MemoryStream.
I tried removing the DeflateStream stuff from both methods, but it's still the same problem. When I look at the examples at MSDN and other places it looks like I'm doing it just right, and googling for the exception message doesn't give any meaningful results(or perhaps I'm just bad at searching).
PS. As you can see I use Miscellaneous.CopyStream(src, dest) which is a basic stream copier, as I can't get src.CopyTo(dest) to work at all, so any hints on that is welcome as well.
Below is a link to the whole VS2010 project if you would like to look at it more closely:
http://www.diredumplings.com/SerializationTesting.zip
UPDATE:
The_Smallest: I tried using the Compress method you posted on my serialization:
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream stream = new MemoryStream())
{
bf.Serialize(stream, profile);
byte[] array = Compress(stream);
using (MemoryStream ms = new MemoryStream(array))
{
using (FileStream fs = File.Create(path))
{
ms.WriteTo(fs);
fs.Flush();
}
}
}
However, it seems to give me the same problems that I had with srcStream.CopyTo(destStream) earlier, which is that it doesn't seem to get written to the stream. The result is a 0 kb file when I try to save it to disk. Any ideas?
Pieter: I removed the MemoryStream from the deserialization method and it seems have the same functionality as before. However I'm not sure how to implement the serialization the way you suggested. Is this what you had in mind?
BinaryFormatter bf = new BinaryFormatter();
using (FileStream fs = File.Create(path))
{
using (DeflateStream ds = new DeflateStream(fs, CompressionMode.Compress))
{
bf.Serialize(ds, profile);
fs.Flush();
ds.Close();
}
fs.Close();
}
Thanks to both of you!
I dowloaded you example and digged a little in there. See changes for your project below:
Replace LoadFromFile in Loader.cs
private static RandomObject LoadFromFile(string path)
{
try
{
var bf = new BinaryFormatter();
using (var fileStream = File.OpenRead(path))
using (var decompressed = new MemoryStream())
{
using (var deflateStream = new DeflateStream(fileStream, CompressionMode.Decompress))
deflateStream.CopyTo(decompressed);
decompressed.Seek(0, SeekOrigin.Begin);
var profile = (RandomObject)bf.Deserialize(decompressed);
profile.SavePath = path;
return profile;
}
}
catch (IOException e)
{
MessageBox.Show(e.Message);
return null;
}
}
Replace Save in Saver.cs as follows:
public static bool Save(RandomObject profile, String path)
{
try
{
var bf = new BinaryFormatter();
using (var uncompressed = new MemoryStream())
using (var fileStream = File.Create(path))
{
bf.Serialize(uncompressed, profile);
uncompressed.Seek(0, SeekOrigin.Begin);
using (var deflateStream = new DeflateStream(fileStream, CompressionMode.Compress))
uncompressed.CopyTo(deflateStream);
}
return true;
}
catch (IOException e)
{
MessageBox.Show(e.Message);
return false;
}
}
You should serialize into the DeflateStream, not the base (MemoryStream) stream.
For serializing: begin with the File.Create. Then around that stream, create the DeflateStream. Then to the DefaulteStream, serialize your objects.
For deserializing: do not create the MemoryStream and deserialize directly from the DeflateStream.
I believe there is no need for the added MemoryStream. If however you do have problems writing directly to/reading directly from the file streams, just change the serialize routine to write to the DeflateStream instead of the MemoryStream.
That should solve your issues.
There is error in streams logics, while compressing you should write to CompressStream, which writes to MemoryStream, after this you will have result in MemoryStream (not in CompressStream)
Here is example how to compress and decompress bytes
private static byte[] Compress(Stream stream)
{
using (var resultStream = new MemoryStream())
{
using (var gzipStream = new DeflateStream(resultStream, CompressionMode.Compress))
stream.CopyTo(gzipStream);
return resultStream.ToArray();
}
}
private static byte[] Decompress(byte[] bytes)
{
using (var readStream = new MemoryStream(bytes))
using (var resultStream = new MemoryStream())
{
using (var gzipStream = new DeflateStream(readStream, CompressionMode.Decompress))
gzipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
}
Related
I need to pass a stream to a couple of functions in another class, but its throwing an error
Cannot access a closed stream
Here's the code:
first method:
Here it opens a file with File.Open method and then creates a memorystream object and it copies FileStream to MemoryStream. then sets Position to 0 (i set position to 0, because i was that in a solution, but not helping tho). Then it creates an object of class DocxConvert and call the Converto method by passing MemoryStream to it.
using (var stream = File.Open(tempPath2, FileMode.Open))
{
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0;
using (var docx = new DocxConvert())
{
return docx.Converto(ms);
}
}
}
DocxConvert Class:
It takes stream and then calls copyStream method by passing the accepted stream.
copyStream method in DocxConvert Class: it should copy the accepted stream to another stream called _memoryStream which is a class property.
public class DocxConvert
{
private MemoryStream _memoryStream = new MemoryStream();
public bool Converto(Stream stream)
{
try
{
copyStream(stream);
//more code
}
catch (IOException ex)
{
Debug.WriteLine(ex);
}
return true;
}
private void copyStream(Stream stream)
{
stream.CopyTo(_memoryStream); //here it throws the error
}
}
p.s. I search for this error here before posting, but non of the topics helped me.
SOLVED by restarting PC, the code it ok.
I am not aware about your question .But here in the code bellow no excepion
private void button1_Click(object sender, EventArgs e)
{
string tempPath2 = Application.StartupPath + "//" + "test.txt";
using (var stream = File.Open(tempPath2, FileMode.Open))
{
using (var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0;
var docx = new DocxConvert();
var isok = docx.Converto(ms);
}
}
}
The bellow is the calss defined where _memorystream is defined at top
MemoryStream _memoryStream = new MemoryStream();
public bool Converto(Stream stream)
{
try
{
copyStream(stream);
//more code
}
catch (IOException ex)
{
// Debug.WriteLine(ex);
}
return true;
}
private void copyStream(Stream stream)
{
try
{
stream.CopyTo(_memoryStream);
}
catch (Exception)
{
}
}
I am trying to use the WCF DataContractSerializer to serialize a DataContract object into a memoryStream.
Then I use the memoryStream.ToArray to get the serialized content.
Finally, I persist the memoryStream into a file using anther fileStream.
My initial implement is like this. I am missing bytes at the end of the persisted File.
public virtual string SerializeTransient(DataObject data, string targetPath)
{
string securityCode;
using (var memoryStream = new MemoryStream())
{
using (var xmlWriter = XmlWriter.Create(memoryStream, new XmlWriterSettings {Indent = true}))
{
_serializer.WriteObject(xmlWriter, data);
using (var fileStream = new FileStream(targetPath, FileMode.Create))
{
securityCode = CalculateSecurityCode(memoryStream.ToArray());
memoryStream.WriteTo(fileStream);
}
}
}
return securityCode;
}
If I move the persist logic out of the inner using{} block (see below), the output is correct. It almost feels like the WriteObject function didnt finish what it is doing. Could someone please explain to me what is happening there? Thanks.
public virtual string SerializeTransient(DataObject data, string targetPath)
{
string securityCode;
using (var memoryStream = new MemoryStream())
{
using (var xmlWriter = XmlWriter.Create(memoryStream, new XmlWriterSettings {Indent = true}))
{
_serializer.WriteObject(xmlWriter, data);
}
using (var fileStream = new FileStream(targetPath, FileMode.Create))
{
securityCode = CalculateSecurityCode(memoryStream.ToArray());
memoryStream.WriteTo(fileStream);
}
}
return securityCode;
}
XmlWriter has an internal buffer. You should either Close/Dispose XmlWriter or call the XmlWriter.Flush() to force all content to be written to underlying stream (memoryStream).
If memoryStream.ToArray() is called before writer.Flush() then some bytes will possibly remain in internal writer buffer.
I built (based on a CodeProject article) a wrapper class (C#) to use a GZipStream to compress a MemoryStream. It compresses fine but doesn't decompress. I've looked at many other examples that have the same problem, and I feel like I'm following what's said but still am getting nothing when I decompress. Here's the compression and decompression methods:
public static byte[] Compress(byte[] bSource)
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Compress, true))
{
gzip.Write(bSource, 0, bSource.Length);
gzip.Close();
}
return ms.ToArray();
}
}
public static byte[] Decompress(byte[] bSource)
{
try
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Decompress, true))
{
gzip.Read(bSource, 0, bSource.Length);
gzip.Close();
}
return ms.ToArray();
}
}
catch (Exception ex)
{
throw new Exception("Error decompressing byte array", ex);
}
}
Here's an example of how I use it:
string sCompressed = Convert.ToBase64String(CompressionHelper.Compress("Some Text"));
// Other Processes
byte[] bReturned = CompressionHelper.Decompress(Convert.FromBase64String(sCompressed));
// bReturned has no elements after this line is executed
There is a bug in Decompress method.
The code does not read content of bSource. On the contrary, it overrides its content wile reading from empty gzip, created based on empty memory stream.
Basically what your version of code is doing:
//create empty memory
using (MemoryStream ms = new MemoryStream())
//create gzip stream over empty memory stream
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Compress, true))
// write from empty stream to bSource
gzip.Write(bSource, 0, bSource.Length);
The fix could look like this:
public static byte[] Decompress(byte[] bSource)
{
using (var inStream = new MemoryStream(bSource))
using (var gzip = new GZipStream(inStream, CompressionMode.Decompress))
using (var outStream = new MemoryStream())
{
gzip.CopyTo(outStream);
return outStream.ToArray();
}
}
The OP said in an edit, now rolled back:
Thanks to Alex's explanation of what was going wrong, I was able to fix the Decompress method. Unfortunately, I'm using .Net 3.5, so I wasn't able to implement the Stream.CopyTo method he suggested. With his explanation, though, I was able to figure out a solution. I made the appropriate changes to the Decompress method below.
public static byte[] Decompress(byte[] bSource)
{
try
{
using (var instream = new MemoryStream(bSource))
{
using (var gzip = new GZipStream(instream, CompressionMode.Decompress))
{
using (var outstream = new MemoryStream())
{
byte[] buffer = new byte[4096];
while (true)
{
int delta = gzip.Read(buffer, 0, buffer.Length);
if (delta > 0)
outstream.Write(buffer, 0, delta);
if (delta < 4096)
break;
}
return outstream.ToArray();
}
}
}
}
catch (Exception ex)
{
throw new Exception("Error decompressing byte array", ex);
}
}
I am currently trying to serialize a List, it serializes (I think fine), but when it deserialize,
Sorry for the amount of code, but I am really stuck and have no idea why this is happening, i also tried to changed the struct into a class and no help.
THANKS.
i get the following error UPDATED
There was an error deserializing the object of type There was an error deserializing the object of type
`System.Collections.Generic.List`1[[A.B.C.DataValues, A.V, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. Unexpected end of file. Following elements are not closed: Time, DataValues, ArrayOfDataValues.`
I am serializing like this UPDATED
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress);
serializer.WriteObject(w, values);
}
_serializedData = stream.ToArray();
}
}
I am deserializing like this UPDATED
public List<DataValues> DeserializeDataValue()
{
if (SerializedData == null || SerializedData.Length == 0)
{
return new List<DataValues> ();
}
else
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream(SerializedData))
{
using (GZipStream decompress = new GZipStream(stream, CompressionMode.Decompress))
{
XmlDictionaryReader r = XmlDictionaryReader.CreateBinaryReader(decompress, XmlDictionaryReaderQuotas.Max);
return serializer.ReadObject(r, true) as List<DataValues>;
}
}
}
}
Properties
private byte[] _serializedData;
[DataMember]
[Browsable(false)]
public byte[] SerializedData
{
get { return _serializedData; }
set { _serializedData = value; }
}
helper Methods
public static byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16 * 1024];
input.Position = 0;
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
Struct
[DataContract(Name = "DataValues", Namespace = "A.B.C")]
public struct DataValues
{
[DataMember]
public DateTime Time { get; set; }
[DataMember]
public Single Value { get; set; }
public DataValues(DateTime dateTime, Single value)
{
Time = dateTime;
Value = value;
}
}
It’s because you are not serialising the object(s) completely. You need to close the stream(s) after writing, especially when using gzip. Recommended practice is to use using:
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress);
serializer.WriteObject(w, values);
}
_serializedData = stream.ToArray();
}
}
Sorry to be late to this question.
The problem with the initial approach was simply that you weren't flushing (read: disposing) the XmlDictionaryWriter.
This should work (note the 2nd using clause):
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
using (XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress))
{
serializer.WriteObject(w, values);
}
Hope this helps someone.
I can get the sample to work by removing the XmlDictionaryReader and instead directly feeding the input/output stream into the DataContractSerializer. It may be a defect in the XmlDictionaryReader for large compressed collections but I'm not sure.
Hope this helps:
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
serializer.WriteObject(compress , values);
}
_serializedData = stream.ToArray();
}
}
public List<DataValues> DeserializeDataValue()
{
if (SerializedData == null || SerializedData.Length == 0)
{
return new List<DataValues> ();
}
else
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream(SerializedData))
{
using (GZipStream decompress = new GZipStream(stream, CompressionMode.Decompress))
{
return serializer.ReadObject(decompress , true) as List<DataValues>;
}
}
}
}
I ran exactly into the same problem and I finally found the solution : the XmlDictionaryWriter needs to be disposed/closed before the Stream you are writing into is itself closed. I discovered that thanks to the thorough example found at http://www.albahari.com/nutshell/ch15.aspx whiche are more complete than the MSDN ones.
In your sample code, that would be :
using (XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress))
{
serializer.WriteObject(w, values);
}
On my own example, using the XmlDictionaryWriter instead of the plain and by default Xml writer only gave me a ~25% decrease in file size but a factor 3 when reading back the object.
I am created a post before " Object to byte not working " .
I fixed problems that users said me , but there is still problem .
Error Message : The constructor to deserialize an object of type 'WindowsFormsApplication1.Form1+Item' was not found.;
void start()
{
Item item = new Item();
item.files.Add(#"test");
byte[] b = ObjectToByteArray(item);
Item k = Desriles(b);
}
[Serializable]
public class Item : ISerializable
{
public Item()
{
files = new List<string>();
Exclude = false;
CleanEmptyFolder = false;
}
public List<string> files;
public string MusicProfileName;
public bool Exclude;
#region ISerializable Members
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("files", files);
info.AddValue("MusicProfileName", MusicProfileName);
info.AddValue("Exclude", Exclude);
}
#endregion
}
public byte[] ObjectToByteArray(object _Object)
{
using (var stream = new MemoryStream())
{
// serialize object
var formatter = new BinaryFormatter();
formatter.Serialize(stream, _Object);
// get a byte array
var bytes = new byte[stream.Length];
using (BinaryReader br = new BinaryReader(stream))
{
bytes = br.ReadBytes(Convert.ToInt32(stream.Length));
}
return bytes;
}
}
public Item Desriles(byte[] items)
{
using (MemoryStream stream = new MemoryStream())
{
stream.SetLength(items.LongLength);
stream.write(items, 0, items.Length);
var formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
object item = formatter.Deserialize(stream); // Here I will get error
return (Item)item;
}
}
The serialization code can't work properly, you forgot to reset the stream back to the beginning. The better mouse trap:
public byte[] ObjectToByteArray(object _Object) {
using (var stream = new MemoryStream()) {
var formatter = new BinaryFormatter();
formatter.Serialize(stream, _Object);
return stream.ToArray();
}
}
The deserialization code can similarly be simplified:
public Item Desriles(byte[] items) {
using (MemoryStream stream = new MemoryStream(items)) {
var formatter = new BinaryFormatter();
return (Item)formatter.Deserialize(stream);
}
}
And you don't need GetObjectData().
In this section:
using (MemoryStream stream = new MemoryStream())
{
stream.SetLength(items.LongLength);
stream.Read(items, 0, items.Length);
[...]
object item = formatter.Deserialize(stream);
it seems you are creating a new, empty memory stream, then attempting to read from it, and Deserialize from it.
Of course it fails. The stream is empty.
abelenky makes a good point, but I don't think your:
public byte[] ObjectToByteArray(object _Object)
works either.
C# Code Snippet - Object to byte array
C# Code Snippet - Byte array to object
Thanks to All :
I found problem : I should base that to ISerializable