Develop Reference

C# MemoryStream.CopyTo(fileStream) adding extra bytes and corrupting OpenXML file

I'm using OpenXML to generate an Excel spreadsheet.
I'm generating the spreadsheet in a MemoryStream; the caller is writing writing out the actual file. For example, my .Net Core controller will return the memory stream as a FileResult. At the moment, I've got a standalone Console mode program that's writing a FileStream.
PROBLEM: I'm getting extra bytes at the end of the file. Since an OpenXml .xlsx file is a .zip file, the extra bytes effectively corrupt the file.
Program.cs:
using (MemoryStream memoryStream = new MemoryStream())
{
OpenXMLGenerate(memoryStream, sampleData);
long msPos = memoryStream.Position; // Position= 1869: Good!
memoryStream.Position = 0;
using (FileStream fs = new FileStream("myfile.xlsx", FileMode.OpenOrCreate))
{
memoryStream.WriteTo(fs);
long fsPos = fs.Position; // Position= 1869: Good!
}
// Myfile.xlsx filesize= 2014, not 1869! Bad!!!
}
When I open the file in 7-Zip, it it says:
Warnings: There are some data after the end of the payload data
Physical Size: 1869
Tail Size: 145
When I try to open it as a .zip file, Windows says:
The Compressed (zipped) folder is invald.
Q: Any idea why I'm getting a 2014 byte file, instead of 1869 bytes?
Q: What can I do about it?

(Documenting per comments.) The issue could be explained by the file replacing an existing file of length 2014 bytes.
Creating a file stream using a mode of FileMode.OpenOrCreate is equivalent to using FileMode.Open if the referenced file exists. If the length of the memory stream is less than the length of the existing file, the existing file will not be truncated to the length of the memory stream; in this case, if N is the length of the memory stream, the first N bytes of the existing file will be overwritten with the contens of the memory stream, and the remaining bytes will persist from the original file.
Creating the file stream with a file mode of FileMode.Create will replace the existing file entirely (if one exists), eliminating any possibility that the new file will contain remnants of the existing file.

What is the simplest way to decompress a ZIP buffer in C#?

When I use zlib in C/C++, I have a simple method uncompress which only requires two buffers and no more else. Its definition is like this:
int uncompress (Bytef *dest, uLongf *destLen, const Bytef *source,
uLong sourceLen);
/*
Decompresses the source buffer into the destination buffer. sourceLen is the byte length of the source buffer. Upon entry,
destLen is the total size of the destination buffer, which must be
large enough to hold the entire uncompressed data. (The size of
the uncompressed data must have been saved previously by the
compressor and transmitted to the decompressor by some mechanism
outside the scope of this compression library.) Upon exit, destLen
is the actual size of the uncompressed data.
uncompress returns Z_OK if success, Z_MEM_ERROR if there was not enough memory, Z_BUF_ERROR if there was not enough room in the output
buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete.
In the case where there is not enough room, uncompress() will fill
the output buffer with the uncompressed data up to that point.
*/
I want to know if C# has a similar way. I checked SharpZipLib FAQ as follows but did not quite understand:
How do I compress/decompress files in memory?
Use a memory stream when creating the Zip stream!
MemoryStream outputMemStream = new MemoryStream();
using (ZipOutputStream zipOutput = new ZipOutputStream(outputMemStream)) {
// Use zipOutput stream as normal
...
You can get the resulting data with memory stream methods ToArray or GetBuffer.
ToArray is the cleaner and easiest to use correctly with the penalty
of duplicating allocated memory. GetBuffer returns a raw buffer raw
and so you need to account for the true length yourself.
See the framework class library help for more information.
I can't figure out if this block of code is for compression or decompression, if outputMemStream meas a compressed stream or an uncompressed stream. I really hope there is a easy-to-understand-way like in zlib. Thanks you very much if you can help me.

Check out the ZipArchive class, which I think has the features you need to accomplish in-memory decompression of zip files.
Assuming you have an array of bytes (byte []) which represent the ZIP file in memory, you have to instantiate a ZipArchive object which will be used to read that array of bytes and interpret them as the ZIP file you whish to load. If you check the ZipArchive class' available constructors in documentation, you will see that they require a stream object from which the data will be read. So, first step would be to convert your byte [] array to a stream that can be read by the constructors, and you can do this by using a MemoryStream object.
Here's an example of how to list all entries inside of a ZIP archive represented in memory as a bytes array:
byte [] zipArchiveBytes = ...; // Read the ZIP file in memory as an array of bytes
using (var inputStream = new MemoryStream(zipArchiveBytes))
using (var zipArchive = new ZipArchive(inputStream, ZipArchiveMode.Read))
{
Console.WriteLine("Listing archive entries...");
foreach (var archiveEntry in zipArchive.Entries)
Console.WriteLine($" {archiveEntry.FullName}");
}
Each file in the ZIP archive will be represented as a ZipArchiveEntry instance. This class offers properties which allow you to retrieve information such as the original length of a file from the ZIP archive, its compressed length, its name, etc.
In order to read a specific file which is contained inside the ZIP file, you can use ZipArchiveEntry.Open(). The following exemplifies how to open a specific file from an archive, if you have its FullName inside the ZIP archive:
ZipArchiveEntry archEntry = zipArchive.GetEntry("my-folder-inside-zip/dog-picture.jpg");
byte[] readResult;
using (Stream entryReadStream = archEntry.Open())
{
using (var tempMemStream = new MemoryStream())
{
entryReadStream.CopyTo(tempMemStream);
readResult = tempMemStream.ToArray();
}
}
This example reads the given file contents, and returns them as an array of bytes (stored in the byte[] readResult variable) which you can then use according to your needs.

7zip compress network stream

I will like to compress a file before sending it through the network. I think the best approach is 7zip because it is free and open source.
How I use 7zip with .net?
I know that 7zip is free and that they have the source code in c# but for some reason it is very slow on c# so I rather call the dll 7z.dll that comes when installing 7zip for performance reasons. So the way I am able to eassily marshal and call the methods in 7z.dll is with the help of the library called sevenzipsharp . For example adding that dll to my project will enable me to do:
// if you installed 7zip 64bit version then make sure you change plataform target
// like on the picture I showed above!
SevenZip.SevenZipCompressor.SetLibraryPath(#"C:\Program Files\7-Zip\7z.dll");
var stream = System.IO.File.OpenRead(#"SomeFileToCompress.txt");
var outputStream = System.IO.File.Create("Output.7z");
SevenZip.SevenZipCompressor compressor = new SevenZip.SevenZipCompressor();
compressor.CompressionMethod = SevenZip.CompressionMethod.Lzma2;
compressor.CompressionLevel = SevenZip.CompressionLevel.Ultra;
compressor.CompressStream(stream, outputStream);
that's how I use 7zip within c#.
Now my question is:
I will like to send a compressed file over the network. I know I could compress it first then send it. The file is 4GB so I will have to wait a long time for it to compress. I will be wasting a lot of space on hard drive. then I will finally be able to send it. I think that is to complicated. I was wondering how it will be possible to send the file meanwhile it is being compressed.
It seems to be a problem with SevenZipSharp:

Have you considered an alternate library - one that doesn't even require 7-Zip to be installed / available?
From the description posted at http://dotnetzip.codeplex.com/ :
creating zip files from stream content, saving to a stream, extracting
to a stream, reading from a stream
Unlike 7-Zip, DotNetZip is designed to work with C# / .Net.
Plenty of examples - including streaming, are available at http://dotnetzip.codeplex.com/wikipage?title=CS-Examples&referringTitle=Examples .
Another option is to use the 7-Zip Command Line Version (7z.exe), and write to/read from standard in/out. This would allow you to use the 7-Zip file format, while also keeping all of the core work in native code (though there likely won't be much of a significant difference).
Looking back at SevenZipSharp:
Since the 0.29 release, streaming is supported.
Looking at http://sevenzipsharp.codeplex.com/SourceControl/changeset/view/59007#364711 :
it seems you'd want this method:
public void CompressStream(Stream inStream, Stream outStream)
Thank you for considering performance here! I think way too many people would do exactly what you're trying to avoid: compress to a temp file, then do something with the temp file.

CompressStream threw an exception. My code is as follows:
public void TestCompress()
{
string fileToCompress = #"C:\Users\gary\Downloads\BD01.DAT";
byte[] inputBytes = File.ReadAllBytes(fileToCompress);
var inputStream = new MemoryStream(inputBytes);
byte[] zipBytes = new byte[38000000]; // this memory size is large enough.
MemoryStream outStream = new MemoryStream(zipBytes);
string compressorEnginePath = #"C:\Engine\7z.dll";
SevenZipCompressor.SetLibraryPath(compressorEnginePath);
compressor = new SevenZip.SevenZipCompressor();
compressor.CompressionLevel = CompressionLevel.Fast;
compressor.CompressionMethod = CompressionMethod.Lzma2;
compressor.CompressStream(inputStream, outputStream);
inputStream.Close();
outputStream.Close();
The exception messages:
Message: Test method Test7zip.UnitTest1.TestCompress threw exception:
SevenZip.SevenZipException: The execution has failed due to the bug in the SevenZipSharp.
Please report about it to http://sevenzipsharp.codeplex.com/WorkItem/List.aspx, post the release number and attach the archive

Getting a "Cannot Read That as a Zip File" Exception while trying to get a stream from an Inner Zip File (a Zip within another Zip)

In C#, I'm using the DotNetZip
I have a zip called "innerZip.zip" which contains some data,
and another zip called "outerZip.zip" which contains the innerZip.
why am i doing it like this ?
well, when setting the password, the password actually applies to individual entries that are added to the archive and not the whole archive, by using this inner/outer combo,
I could set a pass to the whole inner zip because it's an entry of the outer one.
Problem is, well, code speaks better than normal words:
ZipFile outerZip = ZipFile.Read("outerZip.zip");
outerZip.Password = "VeXe";
Stream innerStream = outerZip["innerZip.zip"].OpenReader();
ZipFile innerZip = ZipFile.Read(innerStream); // I'm getting the exception here.
innerZip["Songs\\IronMaiden"].Extract(tempLocation);
why am I getting that exception ?
the inner file is a zip file, so i shouldn't be getting that exception right ?
is there a way to get around this or I just have to extract the inner one from the outer, and then access it ?
Thanx in advance ..

This exception occurs because the CrcCalculatorStream stream that OpenReader creates is not seekable, and ZipFile.Read(Stream) tries to seek while opening the zip file.
The nature of zip compression prevents seeking to a location in the zipped content, the content must be decompressed in order.
One way around this would be to extract the inner zip file to a MemoryStream and then load that via ZipFile.Read.
MemoryStream ms = new MemoryStream();
outerZip["innerZip.zip"].Extract(ms);
ms.Seek(0, SeekOrigin.Begin);
ZipFile innerZip = ZipFile.Read(ms);
innerZip["Songs\\IronMaiden"].Extract(tempLocation);

GZipStream not working

I am using the following C# code to compress a file:
// Open the stream we want to compress
FileStream fs = File.Create(#"C:\Projects\Samples\test\compressed.zip", 0);
// Creates the GZipStream
GZipStream gzip = new GZipStream(fs, CompressionMode.Compress);
// Reading the content to compress
byte[] bytes = File.ReadAllBytes(#"C:\Projects\Samples\samplefile.xml");
// Writing compressed content
gzip.Write(bytes, 0, bytes.Length);
gzip.Close(); // This also closes the FileStream (the underlying stream)
However, when I extract the file from windows explorer the file loses it's extension so instead of samplefile.xml it just becomes samplefile. Same thing happened with .txt file not just .xml file.
Can you help me see what I'm doing wrong?

ok found the problem:
Line 2 has to be as follows:
FileStream fs = File.Create(#"C:\Projects\Samples\test\compressed.xml.zip", 0);

GZipStream doesn't create zip archives. It creates a gzip file, which contains only one file, and doesn't necessarily store a filename at all. Normally you should use the .gz extension to identify a gzip file, and it's conventional to use the entire name of the original file with .gz appended on the end. See also here for more information about gzip format: http://en.wikipedia.org/wiki/Gzip#File_format
If you actually want to create zip archives, you might want to use a library like SharpZipLib: http://www.icsharpcode.net/opensource/sharpziplib/