I need to compress object[][] in C#. I can use Gzip comrepss- decompress a byte[] but how to do it in such cases ?
Use binary serialization to turn it into a byte array which you'd then zip. Provided all the objects in the array are serializable, you can do the following
object[][] objects = new[] {new[] {"a"}};
BinaryFormatter formatter = new BinaryFormatter();
using (MemoryStream memoryStream = new MemoryStream())
using (GZipStream gZipStream = new GZipStream(File.OpenWrite("C:\\zipped.zip"),
CompressionMode.Compress))
{
formatter.Serialize(gZipStream, objects);
}
//unzipping
using (GZipStream gZipStream = new GZipStream(File.OpenRead("C:\\zipped.zip"),
CompressionMode.Decompress))
{
objects = (object[][])formatter.Deserialize(gZipStream);
Console.WriteLine(objects[0][0]); //a
}
Convert your object[][] into a byte[]
Compress the byte[]
Step 1 is the hard part, and will require that all your objects are serializable, so that they can be converted into byte[]s.
The reason this is tricky is that the contents of an object[] are actually just memory references to various objects in memory, and those objects can likewise have references to other objects. Some can even reference specific system resources like I/O ports that have been allocated to them. It wouldn't make sense to send an object like this to a different computer, because that computer has not given the same resources to the object, right? So unless the classes have specifically indicated that they can be serialized and deserialized to a byte stream, you can't do anything with them.
Related
From what I understand, the Formatter converts a serializable object into a stream of bytes and the Stream (e.g. FileStream) does the actual writing of those bytes into a file. Example code:
public static void SaveData(MySerializableData data)
{
BinaryFormatter formatter = new BinaryFormatter();
FileStream stream = new FileStream(SavePath, FileMode.Create);
formatter.Serialize(stream, data);
stream.Close();
}
However at other times, I'm also seeing this type of code:
void Save()
{
string data = "Value 1";
StreamWriter writer = new StreamWriter("MySaveFile.txt", true);
writer.Write(data);
}
Why is it that in the second case, we abandon the 2-step process of saving? Why do we sometimes use only StreamWriter, but at other times use both the formatter and a stream object?
Thank you!
BinaryFormatter serializes a class into a byte array and writes it to a stream. It's useful when you want to save/load classes with it's data. Serializtion stores metadata about the class graph that is storing.
StreamWriter is a stream that has specific functions to write a string into a file.
Consider this example:
MemoryStream mstr = new MemoryStream();
string datastr = "hello!";
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(mstr, datastr);
mstr.Seek(0, SeekOrigin.Begin);
string resultString = Encoding.ASCII.GetString(mstr.ToArray());
If you inspect resultString you will find that it contains something like this:
"\0\u0001\0\0\0????\u0001\0\0\0\0\0\0\0\u0006\u0001\0\0\0\u0006hello!\v"
Well, that's not what you would want to have in a text file, right? As you see serialization is not intended to store raw data but class instances.
Now check this:
MemoryStream mstr = new MemoryStream();
StreamWriter sw = new StreamWriter(mstr);
string datastr = "hello!";
sw.Write(datastr);
mstr.Seek(0, SeekOrigin.Begin);
sw.Close();
string resultString = Encoding.ASCII.GetString(mstr.ToArray());
If you now inspect resultString it will contain:
"hello!"
As you see it's very different, that's what you would expect in a text file.
You can also store raw binary data with a stream:
byte[] data = new byte[]{ 1,2,3,4 };
var fs = File.Create("out.dat"); //this creates a new file and creates a filestream
fs.Write(data, 0, data.Length);
fs.Close();
If you now inspect the file with a binary editor you will see it contains:
0x01, 0x02, 0x03, 0x04
There are many types of streams with different purposes (per example, the MemoryStream that I used in the examples, a binary stream that stores it's data into an array in memory) and a ton of classes that use streams for many things, in this case you have mixed the concepts of serialization and data storage using streams, those are two different things.
I am trying to serialize a class object using binary serialization in C#. I have tried and everywhere all I can find that the serialized data goes to a file always in all the examples I have seen.
In my case, I have to store the serialized data in SQL. The following is an example of the method I have created.
//Serializing the List
public void Serialize(Employees emps, String filename)
{
//Create the stream to add object into it.
System.IO.Stream ms = File.OpenWrite(filename);
//Format the object as Binary
BinaryFormatter formatter = new BinaryFormatter();
//It serialize the employee object
formatter.Serialize(ms, emps);
ms.Flush();
ms.Close();
ms.Dispose();
}
How can I get the serialized data directly in a string variable? I don't want to use a file.
Please help.
The easiest way to represent a byte array as a string in C# is with base64 encoding. The below example shows how this would be achieved within your code.
public void Serialize(Employees emps, String filename)
{
//Create the stream to add object into it.
MemoryStream ms = new MemoryStream();
//Format the object as Binary
BinaryFormatter formatter = new BinaryFormatter();
//It serialize the employee object
formatter.Serialize(ms, emps);
// Your employees object serialised and converted to a string.
string encodedObject = Convert.ToBase64String(ms.ToArray());
ms.Close();
}
This creates the string encodedObject. To retrieve the byte array and your serialised object back from the string you will use the below code.
BinaryFormatter bf = new BinaryFormatter();
// Decode the string back to a byte array
byte[] decodedObject = Convert.FromBase64String(encodedObject);
// Create a memory stream and pass in the decoded byte array as the parameter
MemoryStream memoryStream = new MemoryStream(decodedObject);
// Deserialise byte array back to employees object.
Employees employees = bf.Deserialize(memoryStream);
Just use MemoryStream ms = new MemoryStream() instead of your file stream.
You can extract a byte[] for Storage to SQL after serializing by calling ms.ToArray().
And don't forget to put your Stream into a using-Statement, to guarantee correct disposal of the allocated resources.
I am given an byte array of compressed data, which takes up 27878 bytes. It starts with
78-9C-ED-BD-09
so it should be readable with DeflateStream, which it is. The uncompressed data is a XML file. I want to modify some values in this XML and then compress it again, to save it back to its source.
But even without any modifications, just decompressing and compressing again the result differs from the source, which causes the target-applications, which reads this byte array to crash.
For compression and uncompression I used these methods found in Stackoverflow
private static MemoryStream Decompress(byte[] input)
{
var output = new MemoryStream();
using (var compressStream = new MemoryStream(input))
using (var decompressor = new DeflateStream(compressStream, CompressionMode.Decompress))
decompressor.CopyTo(output);
output.Position = 0;
return output;
}
public byte[] Compress(byte[] input)
{
MemoryStream stream = new MemoryStream();
DeflateStream compressionStream =
new DeflateStream(stream, CompressionMode.Compress);
compressionStream.Write(input, 0, input.Length);
compressionStream.Close();
return stream.ToArray();
}
Edit 04/23/19
As it was pointed out be a comment, that has been deleted, the deflate methods are not suited to compress data. Instead, with the DotNetZip library and some work in this library it was possible to create the same data again!
I'm creating a program which has to send data between a client and server efficiently. To organize packets clearly, I'm using serialization. However, when I serialize these packets the data is unnecessarily large. I'll explain what I'm doing so that you can understand what I need.
My packet classes work like this. I have a Packet object:
using System;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
[Serializable]
public class Packet
{
public static byte[] Serialize(Object o)
{
MemoryStream ms = new MemoryStream();
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(ms, o);
return ms.ToArray();
}
public static Object Deserialize(byte[] bt)
{
MemoryStream ms = new MemoryStream();
BinaryFormatter bf = new BinaryFormatter();
ms.Write(bt, 0, bt.Length);
ms.Position = 0;
object obj = bf.Deserialize(ms);
ms.Close();
return obj;
}
}
I can then create other classes that inherit from the Packet class, here's an example:
using System;
[Serializable]
public class PacketUserInfo : Packet
{
public string Name;
public int Age;
}
Then, it's very simple to put this into a byte array and send it (Of course the above packet is merely an example). However, the size of the resulting array is at least 10 times larger than it would be if I was to use a BinaryWriter and manually write the information.
Why is the serialized data so large? Is there any way to decrease it while still keeping everything organized with packets as their own classes?
Note: I'm only intending to serialize simple properties like this, nothing fancy.
Where you say "Why is the serialized data [...] larger than it would be if I was to use a BinaryWriter and manually write the information", with information you mean property values. The serializer you use however, serializes not only the data, but also some information about the class. You can see this by viewing the serialized data in a text editor.
Is there any way to decrease it while still keeping everything organized with packets as their own classes?
Use more specialized serialization, like protobuf or the library suggested by #Piotr.
Also I think your serialization code should not reside in the Packet base class, but rather in a separate class, like PacketEncoder.
I am trying to format a byte array in C#, by porting a code from Java. In Java, the methods "buf.putInt(value);", buf.putShort, buf.putDouble, (and so forth) are used. However I don't know how to port this to C#. I have tried the MemoryStream class, but there is no method to put a specific type at the end of the byte array.
Question: What is the equivalent of Java's "ByteBuffer.putType(value)" in C#?
Thanks!
You can use a BinaryWriter and your MemoryStream:
MemoryStream stream = new MemoryStream();
using (BinaryWriter writer = new BinaryWriter(stream))
{
writer.Write(myByte);
writer.Write(myInt32);
writer.Write("Hello");
}
byte[] bytes = stream.ToArray();
Try the BinaryWriter class:
using (var binaryWriter = new BinaryWriter(...))
{
binaryWriter.Write(323);
binaryWriter.Write(3487d);
binaryWriter.Write("Hello");
}
You'll be wanting to use the BitConverter class. The main difference is that these methods return an array of bytes instead of altering an existing array.
(This is a replacement for the specific methods mentioned; for a replacement of the entire ByteBuffer class, see the other replies.)