Multithreading compress\decompress with use GzipStream

Multithreading compress\decompress with use GzipStream - c#

I read compressed data one block at a time and add these blocks to the queue. Then when I try to decompress that data using GzipStream InvalidDataException is shown stating that the magic number in the GzipHeader is incorrect. What header should my blocks have?
Here's my decompression code:
private DataBlock Decompress(DataBlock sourceBlock)
{
var decompressed = new byte[sourceBlock.Data.Length];
using (MemoryStream memory = new MemoryStream(sourceBlock.Data))
{
using (GZipStream gzip = new GZipStream(memory, CompressionMode.Decompress))
{
gzip.Read(decompressed, 0, sourceBlock.Data.Length);
}
return new DataBlock(decompressed, sourceBlock.Id);
}
}
Here's my compression code:
private DataBlock Compress(DataBlock sourceBlock)
{
using (MemoryStream memory = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(memory, CompressionMode.Compress))
{
gzip.Write(sourceBlock.Data, 0, sourceBlock.Data.Length);
}
return new DataBlock(memory.ToArray(), sourceBlock.Id);
}
}
Read blocks from file:
private void ReadDataFromFile()
{
if (File.Exists(this.sourceFilePath))
{
try
{
using (var fsSource = new FileStream(this.sourceFilePath, FileMode.Open, FileAccess.Read))
{
while (fsSource.CanRead)
{
var buffer = new byte[this.bufferLength];
int n = fsSource.Read(buffer, 0, buffer.Length);
if (n == 0)
{
break;
}
var block = new DataBlock(buffer, this.currentReadBlockNumber++);
lock (this.innerDataBlockQueue)
{
innerDataBlockQueue.Enqueue(block);
}
this.newBlockDataLoaded?.Invoke(this, EventArgs.Empty);
SpinWait.SpinUntil(() => this.QueueIsFull(this.innerDataBlockQueue) == false);
}
}
}
catch (Exception e)
{
throw e;
}
}
}

Related

.NET Core Created Zip Archive is corrupted [duplicate]

I'm trying to create a ZIP archive with a simple demo text file using a MemoryStream as follows:
using (var memoryStream = new MemoryStream())
using (var archive = new ZipArchive(memoryStream , ZipArchiveMode.Create))
{
var demoFile = archive.CreateEntry("foo.txt");
using (var entryStream = demoFile.Open())
using (var streamWriter = new StreamWriter(entryStream))
{
streamWriter.Write("Bar!");
}
using (var fileStream = new FileStream(#"C:\Temp\test.zip", FileMode.Create))
{
stream.CopyTo(fileStream);
}
}
If I run this code, the archive file itself is created but foo.txt isn't.
However, if I replace the MemoryStream directly with the file stream, the archive is created correctly:
using (var fileStream = new FileStream(#"C:\Temp\test.zip", FileMode.Create))
using (var archive = new ZipArchive(fileStream, FileMode.Create))
{
// ...
}
Is it possible to use a MemoryStream to create the ZIP archive without the FileStream?

Thanks to ZipArchive creates invalid ZIP file, I got:
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("foo.txt");
using (var entryStream = demoFile.Open())
using (var streamWriter = new StreamWriter(entryStream))
{
streamWriter.Write("Bar!");
}
}
using (var fileStream = new FileStream(#"C:\Temp\test.zip", FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
That indicated we need to call Dispose on ZipArchive before we can use it, which as Amir suggests is likely because it writes final bytes like checksum to the archive that makes it complete. But in order not close the stream so we can re-use it after you need to pass true as the third parameter to ZipArchive.

Just another version of zipping without writing any file.
string fileName = "export_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".xlsx";
byte[] fileBytes = here is your file in bytes
byte[] compressedBytes;
string fileNameZip = "Export_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
using (var outStream = new MemoryStream())
{
using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
var fileInArchive = archive.CreateEntry(fileName, CompressionLevel.Optimal);
using (var entryStream = fileInArchive.Open())
using (var fileToCompressStream = new MemoryStream(fileBytes))
{
fileToCompressStream.CopyTo(entryStream);
}
}
compressedBytes = outStream.ToArray();
}

Set the position of the stream to the 0 before copying it to the zip stream.
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("foo.txt");
using (var entryStream = demoFile.Open())
using (var streamWriter = new StreamWriter(entryStream))
{
streamWriter.Write("Bar!");
}
}
using (var fileStream = new FileStream(#"C:\Temp\test.zip", FileMode.Create))
{
memoryStream.Position=0;
memoryStream.WriteTo(fileStream);
}
}

Working solution for MVC
public ActionResult Index()
{
string fileName = "test.pdf";
string fileName1 = "test.vsix";
string fileNameZip = "Export_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
byte[] fileBytes = System.IO.File.ReadAllBytes(#"C:\test\test.pdf");
byte[] fileBytes1 = System.IO.File.ReadAllBytes(#"C:\test\test.vsix");
byte[] compressedBytes;
using (var outStream = new MemoryStream())
{
using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
var fileInArchive = archive.CreateEntry(fileName, CompressionLevel.Optimal);
using (var entryStream = fileInArchive.Open())
using (var fileToCompressStream = new MemoryStream(fileBytes))
{
fileToCompressStream.CopyTo(entryStream);
}
var fileInArchive1 = archive.CreateEntry(fileName1, CompressionLevel.Optimal);
using (var entryStream = fileInArchive1.Open())
using (var fileToCompressStream = new MemoryStream(fileBytes1))
{
fileToCompressStream.CopyTo(entryStream);
}
}
compressedBytes = outStream.ToArray();
}
return File(compressedBytes, "application/zip", fileNameZip);
}

You need to finish writing the memory stream then read the buffer back.
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create))
{
var demoFile = archive.CreateEntry("foo.txt");
using (var entryStream = demoFile.Open())
using (var streamWriter = new StreamWriter(entryStream))
{
streamWriter.Write("Bar!");
}
}
using (var fileStream = new FileStream(#"C:\Temp\test.zip", FileMode.Create))
{
var bytes = memoryStream.GetBuffer();
fileStream.Write(bytes,0,bytes.Length );
}
}

using System;
using System.IO;
using System.IO.Compression;
namespace ConsoleApplication
{
class Program`enter code here`
{
static void Main(string[] args)
{
using (FileStream zipToOpen = new FileStream(#"c:\users\exampleuser\release.zip", FileMode.Open))
{
using (ZipArchive archive = new ZipArchive(zipToOpen, ZipArchiveMode.Update))
{
ZipArchiveEntry readmeEntry = archive.CreateEntry("Readme.txt");
using (StreamWriter writer = new StreamWriter(readmeEntry.Open()))
{
writer.WriteLine("Information about this package.");
writer.WriteLine("========================");
}
}
}
}
}
}

Function to return stream that contain zip file
public static Stream ZipGenerator(List<string> files)
{
ZipArchiveEntry fileInArchive;
Stream entryStream;
int i = 0;
List<byte[]> byteArray = new List<byte[]>();
foreach (var file in files)
{
byteArray.Add(File.ReadAllBytes(file));
}
var outStream = new MemoryStream();
using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
foreach (var file in files)
{
fileInArchive=(archive.CreateEntry(Path.GetFileName(file), CompressionLevel.Optimal));
using (entryStream = fileInArchive.Open())
{
using (var fileToCompressStream = new MemoryStream(byteArray[i]))
{
fileToCompressStream.CopyTo(entryStream);
}
i++;
}
}
}
outStream.Position = 0;
return outStream;
}
If you want , write zip to file stream.
using (var fileStream = new FileStream(#"D:\Tools\DBExtractor\DBExtractor\bin\Debug\test.zip", FileMode.Create))
{
outStream.Position = 0;
outStream.WriteTo(fileStream);
}
`

I'm late to the party, but there are scenarios where you can't access the ZipArchive's constructor to set the leaveOpen parameter and where you don't want the ZIP to be written to disk. In my case, the AsiceArchive class I'm using internally creates a ZipArchive but doesn't set leaveOpen to true.
I created a subclass of Stream that delegates all calls to an inner stream (a few clicks with ReSharper). This class is not disposable, so when the ZipArchive gets disposed, nothing happens to the inner stream.
public class NondisposingStreamWrapper : Stream
{
private readonly Stream _streamImplementation;
public NondisposingStreamWrapper(Stream inner) => _streamImplementation = inner;
public override void Flush() => _streamImplementation.Flush();
public override int Read(byte[] buffer, int offset, int count) => _streamImplementation.Read(buffer, offset, count);
public override long Seek(long offset, SeekOrigin origin) => _streamImplementation.Seek(offset, origin);
public override void SetLength(long value) => _streamImplementation.SetLength(value);
public override void Write(byte[] buffer, int offset, int count) => _streamImplementation.Write(buffer, offset, count);
public override bool CanRead => _streamImplementation.CanRead;
public override bool CanSeek => _streamImplementation.CanSeek;
public override bool CanWrite => _streamImplementation.CanWrite;
public override long Length => _streamImplementation.Length;
public override long Position
{
get => _streamImplementation.Position;
set => _streamImplementation.Position = value;
}
}
Use it like this:
using var memoryStream = new MemoryStream();
var output = new NondisposingStreamWrapper(memoryStream);
using (var archive = new ZipArchive(output, ZipArchiveMode.Create))
{
// add entries to archive
}
memoryStream.Flush();
memoryStream.Position = 0;
// write to file just for testing purposes
File.WriteAllBytes("out.zip", memoryStream.ToArray());

This is the way to convert a entity to XML File and then compress it:
private void downloadFile(EntityXML xml) {
string nameDownloadXml = "File_1.xml";
string nameDownloadZip = "File_1.zip";
var serializer = new XmlSerializer(typeof(EntityXML));
Response.Clear();
Response.ClearContent();
Response.ClearHeaders();
Response.AddHeader("content-disposition", "attachment;filename=" + nameDownloadZip);
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry(nameDownloadXml);
using (var entryStream = demoFile.Open())
using (StreamWriter writer = new StreamWriter(entryStream, System.Text.Encoding.UTF8))
{
serializer.Serialize(writer, xml);
}
}
using (var fileStream = Response.OutputStream)
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
Response.End();
}

Just in case, if anyone wants to save a dynamic zip file through SaveFileDialog.
var logFileName = "zip_filename.zip";
appLogSaver.FileName = logFileName;
appLogSaver.Filter = "LogFiles|*.zip";
appLogSaver.DefaultExt = "zip";
DialogResult resDialog = appLogSaver.ShowDialog();
if (resDialog.ToString() == "OK")
{
System.IO.FileStream fs = (System.IO.FileStream)appLogSaver.OpenFile();
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("foo.txt");
using (var entryStream = demoFile.Open())
{
using (var streamWriter = new StreamWriter(entryStream))
{
//read your existing file and put the content here
streamWriter.Write("Bar!");
}
}
var demoFile2 = archive.CreateEntry("foo2.txt");
using (var entryStream = demoFile2.Open())
{
using (var streamWriter = new StreamWriter(entryStream))
{
streamWriter.Write("Bar2!");
}
}
}
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fs);
}
fs.Close();
}

For me something like this was ok:
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var file = archive.CreateEntry("file.json");
using var entryStream = file.Open();
using var streamWriter = new StreamWriter(entryStream);
streamWriter.WriteLine(someJsonLine);
}
}

private void button6_Click(object sender, EventArgs e)
{
//create With Input FileNames
AddFileToArchive_InputByte(new ZipItem[]{ new ZipItem( #"E:\b\1.jpg",#"images\1.jpg"),
new ZipItem(#"E:\b\2.txt",#"text\2.txt")}, #"C:\test.zip");
//create with input stream
AddFileToArchive_InputByte(new ZipItem[]{ new ZipItem(File.ReadAllBytes( #"E:\b\1.jpg"),#"images\1.jpg"),
new ZipItem(File.ReadAllBytes(#"E:\b\2.txt"),#"text\2.txt")}, #"C:\test.zip");
//Create Archive And Return StreamZipFile
MemoryStream GetStreamZipFile = AddFileToArchive(new ZipItem[]{ new ZipItem( #"E:\b\1.jpg",#"images\1.jpg"),
new ZipItem(#"E:\b\2.txt",#"text\2.txt")});
//Extract in memory
ZipItem[] ListitemsWithBytes = ExtractItems(#"C:\test.zip");
//Choese Files For Extract To memory
List<string> ListFileNameForExtract = new List<string>(new string[] { #"images\1.jpg", #"text\2.txt" });
ListitemsWithBytes = ExtractItems(#"C:\test.zip", ListFileNameForExtract);
// Choese Files For Extract To Directory
ExtractItems(#"C:\test.zip", ListFileNameForExtract, "c:\\extractFiles");
}
public struct ZipItem
{
string _FileNameSource;
string _PathinArchive;
byte[] _Bytes;
public ZipItem(string __FileNameSource, string __PathinArchive)
{
_Bytes=null ;
_FileNameSource = __FileNameSource;
_PathinArchive = __PathinArchive;
}
public ZipItem(byte[] __Bytes, string __PathinArchive)
{
_Bytes = __Bytes;
_FileNameSource = "";
_PathinArchive = __PathinArchive;
}
public string FileNameSource
{
set
{
FileNameSource = value;
}
get
{
return _FileNameSource;
}
}
public string PathinArchive
{
set
{
_PathinArchive = value;
}
get
{
return _PathinArchive;
}
}
public byte[] Bytes
{
set
{
_Bytes = value;
}
get
{
return _Bytes;
}
}
}
public void AddFileToArchive(ZipItem[] ZipItems, string SeveToFile)
{
MemoryStream memoryStream = new MemoryStream();
//Create Empty Archive
ZipArchive archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true);
foreach (ZipItem item in ZipItems)
{
//Create Path File in Archive
ZipArchiveEntry FileInArchive = archive.CreateEntry(item.PathinArchive);
//Open File in Archive For Write
var OpenFileInArchive = FileInArchive.Open();
//Read Stream
FileStream fsReader = new FileStream(item.FileNameSource, FileMode.Open, FileAccess.Read);
byte[] ReadAllbytes = new byte[4096];//Capcity buffer
int ReadByte = 0;
while (fsReader.Position != fsReader.Length)
{
//Read Bytes
ReadByte = fsReader.Read(ReadAllbytes, 0, ReadAllbytes.Length);
//Write Bytes
OpenFileInArchive.Write(ReadAllbytes, 0, ReadByte);
}
fsReader.Dispose();
OpenFileInArchive.Close();
}
archive.Dispose();
using (var fileStream = new FileStream(SeveToFile, FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
public MemoryStream AddFileToArchive(ZipItem[] ZipItems)
{
MemoryStream memoryStream = new MemoryStream();
//Create Empty Archive
ZipArchive archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true);
foreach (ZipItem item in ZipItems)
{
//Create Path File in Archive
ZipArchiveEntry FileInArchive = archive.CreateEntry(item.PathinArchive);
//Open File in Archive For Write
var OpenFileInArchive = FileInArchive.Open();
//Read Stream
FileStream fsReader = new FileStream(item.FileNameSource, FileMode.Open, FileAccess.Read);
byte[] ReadAllbytes = new byte[4096];//Capcity buffer
int ReadByte = 0;
while (fsReader.Position != fsReader.Length)
{
//Read Bytes
ReadByte = fsReader.Read(ReadAllbytes, 0, ReadAllbytes.Length);
//Write Bytes
OpenFileInArchive.Write(ReadAllbytes, 0, ReadByte);
}
fsReader.Dispose();
OpenFileInArchive.Close();
}
archive.Dispose();
return memoryStream;
}
public void AddFileToArchive_InputByte(ZipItem[] ZipItems, string SeveToFile)
{
MemoryStream memoryStream = new MemoryStream();
//Create Empty Archive
ZipArchive archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true);
foreach (ZipItem item in ZipItems)
{
//Create Path File in Archive
ZipArchiveEntry FileInArchive = archive.CreateEntry(item.PathinArchive);
//Open File in Archive For Write
var OpenFileInArchive = FileInArchive.Open();
//Read Stream
// FileStream fsReader = new FileStream(item.FileNameSource, FileMode.Open, FileAccess.Read);
byte[] ReadAllbytes = new byte[4096];//Capcity buffer
int ReadByte = 4096 ;int TotalWrite=0;
while (TotalWrite != item.Bytes.Length)
{
if(TotalWrite+4096>item.Bytes.Length)
ReadByte=item.Bytes.Length-TotalWrite;
Array.Copy(item.Bytes, TotalWrite, ReadAllbytes, 0, ReadByte);
//Write Bytes
OpenFileInArchive.Write(ReadAllbytes, 0, ReadByte);
TotalWrite += ReadByte;
}
OpenFileInArchive.Close();
}
archive.Dispose();
using (var fileStream = new FileStream(SeveToFile, FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
public MemoryStream AddFileToArchive_InputByte(ZipItem[] ZipItems)
{
MemoryStream memoryStream = new MemoryStream();
//Create Empty Archive
ZipArchive archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true);
foreach (ZipItem item in ZipItems)
{
//Create Path File in Archive
ZipArchiveEntry FileInArchive = archive.CreateEntry(item.PathinArchive);
//Open File in Archive For Write
var OpenFileInArchive = FileInArchive.Open();
//Read Stream
// FileStream fsReader = new FileStream(item.FileNameSource, FileMode.Open, FileAccess.Read);
byte[] ReadAllbytes = new byte[4096];//Capcity buffer
int ReadByte = 4096 ;int TotalWrite=0;
while (TotalWrite != item.Bytes.Length)
{
if(TotalWrite+4096>item.Bytes.Length)
ReadByte=item.Bytes.Length-TotalWrite;
Array.Copy(item.Bytes, TotalWrite, ReadAllbytes, 0, ReadByte);
//Write Bytes
OpenFileInArchive.Write(ReadAllbytes, 0, ReadByte);
TotalWrite += ReadByte;
}
OpenFileInArchive.Close();
}
archive.Dispose();
return memoryStream;
}
public void ExtractToDirectory(string sourceArchiveFileName, string destinationDirectoryName)
{
//Opens the zip file up to be read
using (ZipArchive archive = ZipFile.OpenRead(sourceArchiveFileName))
{
if (Directory.Exists(destinationDirectoryName)==false )
Directory.CreateDirectory(destinationDirectoryName);
//Loops through each file in the zip file
archive.ExtractToDirectory(destinationDirectoryName);
}
}
public void ExtractItems(string sourceArchiveFileName,List< string> _PathFilesinArchive, string destinationDirectoryName)
{
//Opens the zip file up to be read
using (ZipArchive archive = ZipFile.OpenRead(sourceArchiveFileName))
{
//Loops through each file in the zip file
foreach (ZipArchiveEntry file in archive.Entries)
{
int PosResult = _PathFilesinArchive.IndexOf(file.FullName);
if (PosResult != -1)
{
//Create Folder
if (Directory.Exists( destinationDirectoryName + "\\" +Path.GetDirectoryName( _PathFilesinArchive[PosResult])) == false)
Directory.CreateDirectory(destinationDirectoryName + "\\" + Path.GetDirectoryName(_PathFilesinArchive[PosResult]));
Stream OpenFileGetBytes = file.Open();
FileStream FileStreamOutput = new FileStream(destinationDirectoryName + "\\" + _PathFilesinArchive[PosResult], FileMode.Create);
byte[] ReadAllbytes = new byte[4096];//Capcity buffer
int ReadByte = 0; int TotalRead = 0;
while (TotalRead != file.Length)
{
//Read Bytes
ReadByte = OpenFileGetBytes.Read(ReadAllbytes, 0, ReadAllbytes.Length);
TotalRead += ReadByte;
//Write Bytes
FileStreamOutput.Write(ReadAllbytes, 0, ReadByte);
}
FileStreamOutput.Close();
OpenFileGetBytes.Close();
_PathFilesinArchive.RemoveAt(PosResult);
}
if (_PathFilesinArchive.Count == 0)
break;
}
}
}
public ZipItem[] ExtractItems(string sourceArchiveFileName)
{
List< ZipItem> ZipItemsReading = new List<ZipItem>();
//Opens the zip file up to be read
using (ZipArchive archive = ZipFile.OpenRead(sourceArchiveFileName))
{
//Loops through each file in the zip file
foreach (ZipArchiveEntry file in archive.Entries)
{
Stream OpenFileGetBytes = file.Open();
MemoryStream memstreams = new MemoryStream();
byte[] ReadAllbytes = new byte[4096];//Capcity buffer
int ReadByte = 0; int TotalRead = 0;
while (TotalRead != file.Length)
{
//Read Bytes
ReadByte = OpenFileGetBytes.Read(ReadAllbytes, 0, ReadAllbytes.Length);
TotalRead += ReadByte;
//Write Bytes
memstreams.Write(ReadAllbytes, 0, ReadByte);
}
memstreams.Position = 0;
OpenFileGetBytes.Close();
memstreams.Dispose();
ZipItemsReading.Add(new ZipItem(memstreams.ToArray(),file.FullName));
}
}
return ZipItemsReading.ToArray();
}
public ZipItem[] ExtractItems(string sourceArchiveFileName,List< string> _PathFilesinArchive)
{
List< ZipItem> ZipItemsReading = new List<ZipItem>();
//Opens the zip file up to be read
using (ZipArchive archive = ZipFile.OpenRead(sourceArchiveFileName))
{
//Loops through each file in the zip file
foreach (ZipArchiveEntry file in archive.Entries)
{
int PosResult = _PathFilesinArchive.IndexOf(file.FullName);
if (PosResult!= -1)
{
Stream OpenFileGetBytes = file.Open();
MemoryStream memstreams = new MemoryStream();
byte[] ReadAllbytes = new byte[4096];//Capcity buffer
int ReadByte = 0; int TotalRead = 0;
while (TotalRead != file.Length)
{
//Read Bytes
ReadByte = OpenFileGetBytes.Read(ReadAllbytes, 0, ReadAllbytes.Length);
TotalRead += ReadByte;
//Write Bytes
memstreams.Write(ReadAllbytes, 0, ReadByte);
}
//Create item
ZipItemsReading.Add(new ZipItem(memstreams.ToArray(),file.FullName));
OpenFileGetBytes.Close();
memstreams.Dispose();
_PathFilesinArchive.RemoveAt(PosResult);
}
if (_PathFilesinArchive.Count == 0)
break;
}
}
return ZipItemsReading.ToArray();
}

How do I zip files in Xamarin for Android?

I have a function that creates a zip file a string array of files passed. The function does succeed in creating the zip file and the zip entry files inside it, but these zip entry files are empty. I've tried a couple of different methods - the function code below is the closest I've gotten to something working:
public static bool ZipFile(string[] arrFiles, string sZipToDirectory, string sZipFileName)
{
if (Directory.Exists(sZipToDirectory))
{
FileStream fNewZipFileStream;
ZipOutputStream zos;
try {
fNewZipFileStream = File.Create(sZipToDirectory + sZipFileName);
zos = new ZipOutputStream(fNewZipFileStream);
for (int i = 0; i < arrFiles.Length; i++) {
ZipEntry entry = new ZipEntry(arrFiles[i].Substring(arrFiles[i].LastIndexOf("/") + 1));
zos.PutNextEntry(entry);
FileStream fStream = File.OpenRead(arrFiles[i]);
BufferedStream bfStrm = new BufferedStream(fStream);
byte[] buffer = new byte[bfStrm.Length];
int count;
while ((count = bfStrm.Read(buffer, 0, 1024)) != -1) {
zos.Write(buffer);
}
bfStrm.Close();
fStream.Close();
zos.CloseEntry();
}
zos.Close();
fNewZipFileStream.Close();
return true;
}
catch (Exception ex)
{
string sErr = ex.Message;
return false;
}
finally
{
fNewZipFileStream = null;
zos = null;
}
}
else
{
return false;
}
}
I think it's got to do with the byte stream handling. I've tried this bit of code that handles the stream but it goes into an infinite loop:
while ((count = fStream.Read(buffer, 0, 1024)) != -1) {
zos.Write(buffer, 0, count);
}
fStream.Close();

I found a solution that is quite simple - I used the ReadAllBytes method of the static File class.
ZipEntry entry = new ZipEntry(arrFiles[i].Substring(arrFiles[i].LastIndexOf("/") + 1));
zos.PutNextEntry(entry);
byte[] fileContents = File.ReadAllBytes(arrFiles[i]);
zos.Write(fileContents);
zos.CloseEntry();

Using Read() on a FileStream returns the amount of bytes read into the stream or 0 if the end of the stream has been reached. It will never return a value of -1.
From MSDN:
The total number of bytes read into the buffer. This might be less than the number of bytes requested if that number of bytes are not currently available, orzero if the end of the stream is reached.
I'd modify your code to the following:
System.IO.FileStream fos = new System.IO.FileStream(sZipToDirectory + sZipFileName, FileMode.Create);
Java.Util.Zip.ZipOutputStream zos = new Java.Util.Zip.ZipOutputStream(fos);
byte[] buffer = new byte[1024];
for (int i = 0; i < arrFiles.Length; i++) {
FileInfo fi = new FileInfo (arrFiles[i]);
Java.IO.FileInputStream fis = new Java.IO.FileInputStream(fi.FullName);
ZipEntry entry = new ZipEntry(arrFiles[i].Substring(arrFiles[i].LastIndexOf("/") + 1));
zos.PutNextEntry(entry);
int count = 0;
while ((count = fis.Read(buffer)) > 0) {
zos.Write(buffer, 0, count);
}
fis.Close();
zos.CloseEntry();
}
This is nearly identical to the code I've used for creating zip archives on Android in the past.

Are you allowed to use SharpZip? It's really easy to use.
Here is a blog post I wrote to extract zip files
private static void upzip(string url)
{
WebClient wc = new WebClient();
wc.DownloadFile(url, "temp.zip");
//unzip
ZipFile zf = null;
try
{
zf = new ZipFile(File.OpenRead("temp.zip"));
foreach (ZipEntry zipEntry in zf)
{
string fileName = zipEntry.Name;
byte[] buffer = new byte[4096];
Stream zipStream = zf.GetInputStream(zipEntry);
using (FileStream streamWriter = File.Create( fileName))
{
StreamUtils.Copy(zipStream, streamWriter, buffer);
}
}
}
finally
{
if (zf != null)
{
zf.IsStreamOwner = true;
zf.Close();
}
}
}

private void ZipFolder(string[] _files, string zipFileName)
{
using var memoryStream = new MemoryStream();
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
foreach (var item in _files)
{
var demoFile = archive.CreateEntry(Path.GetFileName(item));
using var readStreamW = File.OpenRead(item);
using (var entryStream = demoFile.Open())
{
using (var streamWriter = new StreamWriter(entryStream))
{
readStreamW.Seek(0, SeekOrigin.Begin);
readStreamW.CopyTo(streamWriter.BaseStream);
}
}
}
}
using var fileStream = new FileStream(zipFileName, FileMode.Create);
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}

System.OutofMemoryException throw while doing GZipStream Compression

I am working in win forms. Getting errors while doing following operation.
It shows me System.OutOfMemoryException error when i try to run the operation around 2-3 times continuously. Seems .NET is not able to free the resouces used in operation. The file i am using for operation is quite big, around more than 500 MB.
My sample code is as below. Please help me how to resolve the error.
try
{
using (FileStream target = new FileStream(strCompressedFileName, FileMode.Create, FileAccess.Write))
using (GZipStream alg = new GZipStream(target, CompressionMode.Compress))
{
byte[] data = File.ReadAllBytes(strFileToBeCompressed);
alg.Write(data, 0, data.Length);
alg.Flush();
data = null;
}
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}

Replace ReadAllBytes with Stream.CopyTo
using (FileStream target = new FileStream(strCompressedFileName, FileMode.Create, FileAccess.Write))
using (GZipStream alg = new GZipStream(target, CompressionMode.Compress))
{
using (var fileToRead = File.Open(.....))
{
fileToRead.CopyTo(alg);
}
}

A very rough example could be
// destFile - FileStream for destinationFile
// srcFile - FileStream of sourceFile
using (GZipStream gz = new GZipStream(destFile, CompressionMode.Compress))
{
byte[] src = new byte[1024];
int count = sourceFile.Read(src, 0, 1024);
while (count != 0)
{
gz.Write(src, 0, count );
count = sourceFile.Read(src, 0, 1024);
}
}
// flush, close, dispose ..
So basically I changed your ReadAllBytes to read only chunks of 1024 bytes.

You can try to use this method to compress file MSDN link
public static void Compress(FileInfo fileToCompress)
{
using (FileStream originalFileStream = fileToCompress.OpenRead())
{
using (FileStream compressedFileStream = File.Create(fileToCompress.FullName + ".gz"))
{
using (GZipStream compressionStream = new GZipStream(compressedFileStream, CompressionMode.Compress))
{
originalFileStream.CopyTo(compressionStream);
}
}
}
}
usage:
string directoryPath = #"c:\users\public\reports";
DirectoryInfo directorySelected = new DirectoryInfo(directoryPath);
foreach (FileInfo fileToCompress in directorySelected.GetFiles())
{
Compress(fileToCompress);
}

Get rid of CA2202

How can I get rid of the CA2202 warning (CA2202 : Microsoft.Usage : Object 'compressedStream' can be disposed more than once in method 'Compression.InternalDecompress(byte[])'. To avoid generating a System.ObjectDisposedException you should not call Dispose more than one time on an object) from the following code:
using (var compressedStream = new MemoryStream(inputData))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
I have tried getting rid of the "using" statement and replacing it with try/finally pattern but then I get CA2000 (CA2000 : Microsoft.Reliability : In method 'Compression.InternalDecompress(byte[])', call System.IDisposable.Dispose on object 'stream' before all references to it are out of scope). I have tried replacing the above code like this:
MemoryStream decompressedData = null;
MemoryStream stream = null;
GZipStream decompressor = null;
try
{
decompressedData = new MemoryStream();
stream = new MemoryStream(inputData);
decompressor = new GZipStream(stream, CompressionMode.Decompress, false);
stream = null;
int bytesRead = 1;
int chunkSize = 4096;
byte[] chunk = new byte[chunkSize];
while ((bytesRead = decompressor.Read(chunk, 0, chunkSize)) > 0)
{
decompressedData.Write(chunk, 0, bytesRead);
}
decompressor = null;
return decompressedData.ToArray();
}
finally
{
if (stream != null)
{
stream.Dispose();
}
if (decompressor != null)
{
decompressor.Dispose();
}
if (decompressedData != null)
{
decompressedData.Dispose();
}
}

This is what i use:
public class Compression
{
public Compression()
{
}
public byte[] Compress(byte[] buffer)
{
byte[] gzBuffer;
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream zip = new GZipStream(ms, CompressionMode.Compress, true))
{
zip.Write(buffer, 0, buffer.Length);
zip.Close();
}
ms.Position = 0;
MemoryStream outStream = new MemoryStream();
byte[] compressed = new byte[ms.Length];
ms.Read(compressed, 0, compressed.Length);
gzBuffer = new byte[compressed.Length + 4];
Buffer.BlockCopy(compressed, 0, gzBuffer, 4, compressed.Length);
Buffer.BlockCopy(BitConverter.GetBytes(buffer.Length), 0, gzBuffer, 0, 4);
}
return gzBuffer;
}
public byte[] Decompress(byte[] gzBuffer)
{
byte[] buffer;
using (MemoryStream ms = new MemoryStream())
{
int msgLength = BitConverter.ToInt32(gzBuffer, 0);
ms.Write(gzBuffer, 4, gzBuffer.Length - 4);
buffer = new byte[msgLength];
ms.Position = 0;
using (GZipStream zip = new GZipStream(ms, CompressionMode.Decompress))
{
zip.Read(buffer, 0, buffer.Length);
}
}
return buffer;
}
}
Or you can just add a pragma statement to your class
#pragma warning disable 2202
namespace Your.Namespace
{
...
}
#pragma warning restore 2202

This is what I ended up using - gets rid of both CA2000 and CA2202:
private static MemoryStream GetMemoryStream()
{
return new MemoryStream();
}
private static byte[] InternalDecompress(byte[] inputData)
{
Debug.Assert(inputData != null, "inputData cannot be null");
MemoryStream decompressedData = GetMemoryStream();
MemoryStream inputDataMemoryStream = GetMemoryStream();
GZipStream decompressor = null;
try
{
inputDataMemoryStream.Write(inputData, 0, inputData.Length);
inputDataMemoryStream.Position = 0;
decompressor = new GZipStream(inputDataMemoryStream, CompressionMode.Decompress, false);
int bytesRead;
int chunkSize = 4096;
byte[] chunk = new byte[chunkSize];
while ((bytesRead = decompressor.Read(chunk, 0, chunkSize)) > 0)
{
decompressedData.Write(chunk, 0, bytesRead);
}
}
finally
{
if (decompressor != null)
{
decompressor.Dispose();
}
}
return decompressedData.ToArray();
}

This is my attempt. It works and it avoids CA2202
/// <summary>
/// Compresses byte array to new byte array.
/// </summary>
public byte[] Compress(byte[] raw)
{
MemoryStream outStream = null;
GZipStream tinyStream = null;
byte[] retValue = null;
try
{
outStream = new MemoryStream();
tinyStream = new GZipStream(outStream, CompressionMode.Compress);
using (var mStream = new MemoryStream(raw))
mStream.CopyTo(tinyStream);
}
finally
{
if (tinyStream != null)
{
tinyStream.Dispose();
retValue = outStream.ToArray();
}
else if (outStream != null)
{
retValue = outStream.ToArray();
outStream.Dispose();
}
}
return retValue;
}

Programmatic compression/decompression to MemoryStream with GZipStream

I built (based on a CodeProject article) a wrapper class (C#) to use a GZipStream to compress a MemoryStream. It compresses fine but doesn't decompress. I've looked at many other examples that have the same problem, and I feel like I'm following what's said but still am getting nothing when I decompress. Here's the compression and decompression methods:
public static byte[] Compress(byte[] bSource)
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Compress, true))
{
gzip.Write(bSource, 0, bSource.Length);
gzip.Close();
}
return ms.ToArray();
}
}
public static byte[] Decompress(byte[] bSource)
{
try
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Decompress, true))
{
gzip.Read(bSource, 0, bSource.Length);
gzip.Close();
}
return ms.ToArray();
}
}
catch (Exception ex)
{
throw new Exception("Error decompressing byte array", ex);
}
}
Here's an example of how I use it:
string sCompressed = Convert.ToBase64String(CompressionHelper.Compress("Some Text"));
// Other Processes
byte[] bReturned = CompressionHelper.Decompress(Convert.FromBase64String(sCompressed));
// bReturned has no elements after this line is executed

There is a bug in Decompress method.
The code does not read content of bSource. On the contrary, it overrides its content wile reading from empty gzip, created based on empty memory stream.
Basically what your version of code is doing:
//create empty memory
using (MemoryStream ms = new MemoryStream())
//create gzip stream over empty memory stream
using (GZipStream gzip = new GZipStream(ms, CompressionMode.Compress, true))
// write from empty stream to bSource
gzip.Write(bSource, 0, bSource.Length);
The fix could look like this:
public static byte[] Decompress(byte[] bSource)
{
using (var inStream = new MemoryStream(bSource))
using (var gzip = new GZipStream(inStream, CompressionMode.Decompress))
using (var outStream = new MemoryStream())
{
gzip.CopyTo(outStream);
return outStream.ToArray();
}
}

The OP said in an edit, now rolled back:
Thanks to Alex's explanation of what was going wrong, I was able to fix the Decompress method. Unfortunately, I'm using .Net 3.5, so I wasn't able to implement the Stream.CopyTo method he suggested. With his explanation, though, I was able to figure out a solution. I made the appropriate changes to the Decompress method below.
public static byte[] Decompress(byte[] bSource)
{
try
{
using (var instream = new MemoryStream(bSource))
{
using (var gzip = new GZipStream(instream, CompressionMode.Decompress))
{
using (var outstream = new MemoryStream())
{
byte[] buffer = new byte[4096];
while (true)
{
int delta = gzip.Read(buffer, 0, buffer.Length);
if (delta > 0)
outstream.Write(buffer, 0, delta);
if (delta < 4096)
break;
}
return outstream.ToArray();
}
}
}
}
catch (Exception ex)
{
throw new Exception("Error decompressing byte array", ex);
}
}

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

Multithreading compress\decompress with use GzipStream - c#

Related

.NET Core Created Zip Archive is corrupted [duplicate]

How do I zip files in Xamarin for Android?

System.OutofMemoryException throw while doing GZipStream Compression

Get rid of CA2202

Programmatic compression/decompression to MemoryStream with GZipStream

Categories

Resources