I am working on an existing system where data is stored in a compressed byte array in a database.
The existing data has all been compressed using GZipDotNet.dll.
I am trying to switch to using the gzip functions in System.IO.Compression.
When I use:
public static byte[] DeCompressByteArray(byte[] inArray)
{
byte[] outStream = null;
outStream = GZipDotNet.GZip.Uncompress(inArray);
return outStream;
}
It works fine but:
public static byte[] DeCompressByteArray(byte[] inArray)
{
byte[] outStream = null;
using (var compressedStream = new MemoryStream(inArray))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
outStream = resultStream.ToArray();
}
return outStream;
}
Gives a response of:
The magic number in GZip header is not correct. Make sure you are passing in a GZip stream
Related
I am trying to decode binary data that represents a gzip file, I need to decompress the gzip so I can get the nbt (minecraft notation thing) string that is in the gzip, but I keep getting the following error at GZipStream.Read:
The archive entry was compressed using an unsupported compression method.
Does anyone have any idea on how to do this?
This is my code:
public static string Decompress(string input)
{
byte[] compressed = Convert.FromBase64String(input); //This is the binary data
byte[] decompressed = Decompress(compressed);
return Encoding.UTF8.GetString(decompressed);
}
private static byte[] Decompress(byte[] input)
{
using (var source = new MemoryStream(input))
{
byte[] lengthBytes = new byte[4];
source.Read(lengthBytes, 0, 4);
var length = BitConverter.ToInt32(lengthBytes, 0);
using (var decompressionStream = new GZipStream(source,
CompressionMode.Decompress))
{
var result = new byte[length];
decompressionStream.Read(result, 0, length); //Error
return result;
}
}
}
This should do it for you:
public static string Decompress(string value)
{
byte[] buffer = Convert.FromBase64String(value);
byte[] decompressed;
using (var inputStream = new MemoryStream(buffer))
{
using var outputStream = new MemoryStream();
using (var gzip = new GZipStream(inputStream, CompressionMode.Decompress, leaveOpen: true))
{
gzip.CopyTo(outputStream);
}
decompressed = outputStream.ToArray();
}
return Encoding.UTF8.GetString(decompressed);
}
I have a very simple gzip method:
public byte[] Compress(string input)
{
var bytes = Encoding.UTF8.GetBytes(input);
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
using (var gz = new GZipStream(mso, CompressionMode.Compress))
{
msi.CopyTo(gz);
return mso.ToArray();
}
}
However, unit tests fail. Even passing in a simple short string doesn't get gzipp'ed properly. e.g. "this is a test" becomes a byte array with 10 elements: [31,139,8,0,0,0,0,0,4,0] which of course doesn't ungzip properly. What's going wrong here? This has come straight from msdn!
You need to flush close the stream for it to compress. At the point you call mso.ToArray(), the GZipStream hasn't compressed anything yet and is waiting for more data.
A simple solution:
public byte[] Compress(string input)
{
var bytes = Encoding.UTF8.GetBytes(input);
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
{
using (var gz = new GZipStream(mso, CompressionMode.Compress))
{
msi.CopyTo(gz);
}
return mso.ToArray();
}
}
Following various samples I've been able to convert a memory stream to a compressed stream and then to a byte array to save in a database but I'm having trouble going the other way. Here's what I've got so far...
...
using (MemoryStream compressedStream = new MemoryStream()) {
...some code that builds the compressedStream for an undetermined
number of byteArrays from a database
using (MemoryStream uncompressedStream = new MemoryStream()) {
// method 1
using (GZipStream unzippedStream = new GZipStream(compressedStream, CompressionMode.Decompress)) {
unzippedStream.CopyTo(uncompressedStream);
}
// method 2
using (GZipStream unzippedStream = new GZipStream(uncompressedStream, CompressionMode.Decompress)) {
compressedStream.CopyTo(unzippedStream);
}
... do something with uncompressedStream
}
}
Method 1 seams to follows the examples I see on here but causes an error "stream does not support writing"
Method 2 seams to make more sense but the uncompressed stream is always empty
P.S. Really what I would like to have is something simple like
MemoryStream compressed = GZipStream(uncompressed, Compress)
MemoryStream upcompressed = GZipStream(compressed, Decompress)
This code example works. The first part is just to get a compressed byte array. The second part demonstrates how the compressed stream can be created in code, write can be done multiple times. But the position must be set to 0.
byte[] compressed;
string output;
using (var outStream = new MemoryStream()) {
using (var tinyStream = new GZipStream(outStream, CompressionMode.Compress))
using (var mStream = new MemoryStream(Encoding.UTF8.GetBytes("This is a test"))) {
mStream.CopyTo(tinyStream);
}
compressed = outStream.ToArray();
}
using (var compressedStream = new MemoryStream()) {
// can do multiple writes here to create the compressed stream
compressedStream.Write(compressed, 0, compressed.Length);
compressedStream.Flush();
compressedStream.Position = 0;
using (var unzippedStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var uncompressedStream = new MemoryStream()) {
unzippedStream.CopyTo(uncompressedStream);
output = Encoding.UTF8.GetString(uncompressedStream.ToArray());
}
}
Console.WriteLine(output);
I am trying to decompress a byte array and get it into a string using a binary reader. When the following code executes, the inStream position changes from 0 to the length of the array, but str is always an empty string.
BinaryReader br = null;
string str = String.Empty;
using (MemoryStream inStream = new MemoryStream(pByteArray))
{
GZipStream zipStream = new GZipStream(inStream, CompressionMode.Decompress);
BinaryReader br = new BinaryReader(zipStream);
str = br.ReadString();
inStream.Close();
br.Close();
}
You haven't shown how is the data being compressed, but here's a full example of compressing and decompressing a buffer:
using System;
using System.IO;
using System.IO.Compression;
using System.Text;
class Program
{
static void Main()
{
var test = "foo bar baz";
var compressed = Compress(Encoding.UTF8.GetBytes(test));
var decompressed = Decompress(compressed);
Console.WriteLine(Encoding.UTF8.GetString(decompressed));
}
static byte[] Compress(byte[] data)
{
using (var compressedStream = new MemoryStream())
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Compress))
{
zipStream.Write(data, 0, data.Length);
zipStream.Close();
return compressedStream.ToArray();
}
}
static byte[] Decompress(byte[] data)
{
using (var compressedStream = new MemoryStream(data))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
}
}
So here's a strange one. I have this method to take a Base64-encoded deflated string and return the original data:
public static string Base64Decompress(string base64data)
{
byte[] b = Convert.FromBase64String(base64data);
using (var orig = new MemoryStream(b))
{
using (var inflate = new MemoryStream())
{
using (var ds = new DeflateStream(orig, CompressionMode.Decompress))
{
ds.CopyTo(inflate);
return Encoding.ASCII.GetString(inflate.ToArray());
}
}
}
}
This returns an empty string unless I add a second call to ds.CopyTo(inflate). (WTF?)
...
using (var ds = new DeflateStream(orig, CompressionMode.Decompress))
{
ds.CopyTo(inflate);
ds.CopyTo(inflate);
return Encoding.ASCII.GetString(inflate.ToArray());
}
...
(Flush/Close/Dispose on ds have no effect.)
Why does the DeflateStream copy 0 bytes on the first call? I've also tried looping with Read(), but it also returns zero on the first call, then works on the second.
Update: here's the method I'm using to compress data.
public static string Base64Compress(string data, Encoding enc)
{
using (var ms = new MemoryStream())
{
using (var ds = new DeflateStream(ms, CompressionMode.Compress))
{
byte[] b = enc.GetBytes(data);
ds.Write(b, 0, b.Length);
ds.Flush();
return Convert.ToBase64String(ms.ToArray());
}
}
}
This happens when the compressed bytes are incomplete (i.e., not all blocks are written out).
If I use your Base64Compress with the following Decompress method I will get an InvalidDataException with the message 'Unknown block type. Stream might be corrupted.'
Decompress
public static string Decompress(Byte[] bytes)
{
using (var uncompressed = new MemoryStream())
using (var compressed = new MemoryStream(bytes))
using (var ds = new DeflateStream(compressed, CompressionMode.Decompress))
{
ds.CopyTo(uncompressed);
return Encoding.ASCII.GetString(uncompressed.ToArray());
}
}
Note that everything works as expected when using the following Compress method
public Byte[] Compress(Byte[] bytes)
{
using (var memoryStream = new MemoryStream())
{
using (var deflateStream = new DeflateStream(memoryStream, CompressionMode.Compress))
deflateStream.Write(bytes, 0, bytes.Length);
return memoryStream.ToArray();
}
}
Update
Oops, foolish me... you cannot ToArray the memory stream until you dispose the DeflateStream (as flush is acutally not implemented (and Deflate/GZip compress blocks of data); the final block is only written on close/dispose.
Re-write compress as:
public static string Base64Compress(string data, Encoding enc)
{
using (var ms = new MemoryStream())
{
using (var ds = new DeflateStream(ms, CompressionMode.Compress))
{
byte[] b = enc.GetBytes(data);
ds.Write(b, 0, b.Length);
}
return Convert.ToBase64String(ms.ToArray());
}
}