Develop Reference

Extract tgz file in memory and access files in C#

I have a service that downloads a *.tgz file from a remote endpoint. I use SharpZipLib to extract and write the content of that compressed archive to disk. But now I want to prevent writing the files to disk (because that process doesn't have write permissions on that disk) and keep them in memory.
How can I access the decompressed files from memory? (Let's assume the archive holds simple text files)
Here is what I have so far:
public void Decompress(byte[] byteArray)
{
Stream inStream = new MemoryStream(byteArray);
Stream gzipStream = new GZipInputStream(inStream);
TarArchive tarArchive = TarArchive.CreateInputTarArchive(gzipStream);
tarArchive.ExtractContents(#".");
tarArchive.Close();
gzipStream.Close();
inStream.Close();
}

Check this and this out.
Turns out, ExtractContents() works by iterating over TarInputStream. When you create your TarArchive like this:
TarArchive.CreateInputTarArchive(gzipStream);
it actually wraps the stream you're passing into a TarInputStream. Thus, if you want more fine-grained control over how you extract files, you must use TarInputStream directly.
See, if you can iterate over files, directories and actual file contents like this:
Stream inStream = new MemoryStream(byteArray);
Stream gzipStream = new GZipInputStream(inStream);
using (var tarInputStream = new TarInputStream(gzipStream))
{
TarEntry entry;
while ((entry = tarInputStream.GetNextEntry()) != null)
{
var fileName = entry.Name;
using (var fileContents = new MemoryStream())
{
tarInputStream.CopyEntryContents(fileContents);
// use entry, fileName or fileContents here
}
}
}

ZipArchive created with C# does not contain any entries

I am attempting to create a zipfile in ASP.NET MVC a single PDF file within it. However, using the code below, an empty zipfile is created. Can someone please advise what I am doing incorrectly?
public FileResult DownloadZipfile(string html)
{
MemoryStream memoryStream = new MemoryStream();
ZipArchive archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true);
byte[] rawDownload = PDFConverterUtils.PdfSharpConvert(html);
ZipArchiveEntry entry = archive.CreateEntry("MyPDF.pdf");
using (Stream entryStream = entry.Open())
using (StreamWriter streamWriter = new StreamWriter(entryStream))
{
streamWriter.BaseStream.Write(rawDownload, 0, rawDownload.Length);
}
return new FileStreamResult(memoryStream, System.Net.Mime.MediaTypeNames.Application.Zip) { FileDownloadName = "test.zip" };
}

When using a ZipArchive with a MemoryStream, I would suggest resetting the position of the stream after writing to it so that the content of the stream can be read by the response.
public FileResult DownloadZipfile(string html) {
byte[] rawDownload = PDFConverterUtils.PdfSharpConvert(html);
MemoryStream memoryStream = new MemoryStream();
using(ZipArchive archive = new ZipArchive(
stream: memoryStream,
mode: ZipArchiveMode.Create,
leaveOpen: true //To leave the memory stream open after disposal
)){
ZipArchiveEntry entry = archive.CreateEntry("MyPDF.pdf");
using (Stream entryStream = entry.Open()) {
entryStream.Write(rawDownload, 0, rawDownload.Length);
}
}
memoryStream.Position = 0;//reset memory stream position for read
return new FileStreamResult(memoryStream, System.Net.Mime.MediaTypeNames.Application.Zip) {
FileDownloadName = "test.zip"
};
}
As suggested in another answer you should dispose of the archive to force it to write its content to its underlying memory stream, but take note of the following
ZipArchive.Dispose()
Unless you construct the object by using the ZipArchive(Stream, ZipArchiveMode, Boolean) constructor overload and set its leaveOpen parameter to true, all underlying streams are closed and no longer available for subsequent write operations.
When you are finished using this instance of ZipArchive, call Dispose() to release all resources used by this instance. You should eliminate further references to this ZipArchive instance so that the garbage collector can reclaim the memory of the instance instead of keeping it alive for finalization.
Because you want to make use of the memory stream after writing to it, you need to make sure that it remains open, and that the position of the stream is reset to the beginning so that the content of the stream can be read from the start.

Disposable that depends on other Disposables

I have an application that processes file streams based on a list of strings, and the string can either be a file on disk, or a file inside a Zip file. To clean up the code, I'd like to refactor out the process of opening the file.
I've created a method that returns a Stream of the file contents, but because the stream depends on the ZipFile IDisposable, by the time I read the stream, the ZipFile is disposed an throws an exception.
void Main()
{
using (var stream = OpenFileForImport("zipfile.zip;insidefile.txt"))
new StreamReader(stream).ReadToEnd(); // Exception
using (var stream = OpenFileForImport("outside.txt"))
new StreamReader(stream).ReadToEnd(); // Works
}
public static Stream OpenFileForImport(string filePath)
{
var path = Path.Combine(basefolder, filePath);
if (path.Contains(";"))
{
var parts = path.Split(';');
var zipPath = parts[0];
//Error checking logic to ensure zip file exists and is valid...
using (var zip = ZipFile.OpenRead(zipPath))
using (var entry = zip.GetEntry(parts[1]))
{
//Error checking logic to ensure inside file exists within zip file.
return entry.Open();
}
}
var file = new FileInfo(path);
if (file != null)
return file.OpenRead();
return null;
}
I could remove the using clause from the zip and entry declarations, but I doubt they'd ever get disposed. Is there an appropriate pattern to return a disposable, when it depends on other disposables?

Don't return the stream directly, instead return a disposable object which can provide the stream you want to dispose, but that cleans up that stream and the other dependant resources when it is disposed of:
public class NameToBeDetermined : IDisposable
{
private ZipFile zip;
public Stream Stream { get; }
public NameToBeDetermined(ZipFile zip, Stream stream)
{
this.zip = zip;
Stream = stream;
}
public void Dispose()
{
zip.Dispose();
Stream.Dispose();
}
}
Then return that, rather than the stream itself. If it's worth spending the time, you could turn that wrapper into a Stream itself, that just forwards all Stream methods into the composed stream, but that does the extra work when disposing. Whether it's worth the time to create that more involved wrapper rather than having a caller access a Stream property is up to you.

You likely should copy the file from the ZipEntry into a MemoryStream so that you have a copy to work with.
//Error checking logic to ensure zip file exists and is valid...
using (var zip = ZipFile.OpenRead(zipPath))
using (var entry = zip.GetEntry(parts[1]))
{
//Error checking logic to ensure inside file exists within zip file.
MemoryStream stream = new MemoryStream();
entry.Open().CopyTo(stream);
stream.Seek(0, SeekOrigin.Begin);
return stream;
}

Zip File getting corrupted when i save it to isolated storage

I have downloaded a zip file from blob storage and save it to isolated storage of windows phone like this :- FileStream fs is downloaded from blob.
public static void SaveToIsolatedStorage(FileStream fs, string fileName)
{
var isolatedStorage = IsolatedStorageFile.GetUserStoreForApplication();
using (var streamWriter =
new StreamWriter(new IsolatedStorageFileStream(fileName,
FileMode.Create,
FileAccess.ReadWrite,
isolatedStorage)))
{
streamWriter.Write(fs);
}
}
But when checked this zip file using IsoStoreSpy it is showing corrupted. I have checked it by reading from isolated storage and tried to unzip it but not working. I am sure that it is corrupted because when i replace this file using IsoStoreSpy with some other zip and then tried to unzip it then it is working.
Edit:-
Code for downloading from Blob
private async Task DownloadFileFromBlobStorage()
{
var filename = "AppId_2.zip";
var blobContainer = GetBlobClient.GetContainerReference("testwpclientiapcontainer");
var blob = blobContainer.GetBlockBlobReference(filename);
using (var filestream = new FileStream(filename, FileMode.Create))
{
await blob.DownloadToStreamAsync(filestream);
SaveToIsolatedStorage(filestream, filename);
}
}
So anybody know how can i save the zip file to isolated storage without getting it corrupted ?

You're using a StreamWriter. That's for text. You shouldn't be using it to copy a zip file at all. Never use any TextWriter for binary data.
Next you're using StreamWriter.Write(object), which is basically going to call ToString on the FileStream. That's not going to work either.
You should just create an IsolatedStorageStream, and then call fs.CopyTo(output).
public static void SaveToIsolatedStorage(Stream input, string fileName)
{
using (var storage = IsolatedStorageFile.GetUserStoreForApplication())
{
// Simpler than calling the IsolatedStorageFileStream constructor...
using (var output = storage.CreateFile(fileName))
{
input.CopyTo(output);
}
}
}
In your edit you've shown code which saves to a FileStream first, and then copies the stream from the current position. As you've noted in comments, you needed to rewind it first.
Personally I wouldn't use a FileStream at all here - why do you want to save it as a normal file and an isolated file? Just use a MemoryStream:
using (var stream = new MemoryStream())
{
await blob.DownloadToStreamAsync(filestream);
stream.Position = 0;
SaveToIsolatedStorage(stream, filename);
}
(Note that your SaveToIsolatedStorage method is still synchronous... you may wish to consider an asynchronous version.)

SharpZipLib Examine and select contents of a ZIP file

I am using SharpZipLib in a project and am wondering if it is possible to use it to look inside a zip file, and if one of the files within has a data modified in a range I am searching for then to pick that file out and copy it to a new directory? Does anybody know id this is possible?

Yes, it is possible to enumerate the files of a zip file using SharpZipLib. You can also pick files out of the zip file and copy those files to a directory on your disk.
Here is a small example:
using (var fs = new FileStream(#"c:\temp\test.zip", FileMode.Open, FileAccess.Read))
{
using (var zf = new ZipFile(fs))
{
foreach (ZipEntry ze in zf)
{
if (ze.IsDirectory)
continue;
Console.Out.WriteLine(ze.Name);
using (Stream s = zf.GetInputStream(ze))
{
byte[] buf = new byte[4096];
// Analyze file in memory using MemoryStream.
using (MemoryStream ms = new MemoryStream())
{
StreamUtils.Copy(s, ms, buf);
}
// Uncomment the following lines to store the file
// on disk.
/*using (FileStream fs = File.Create(#"c:\temp\uncompress_" + ze.Name))
{
StreamUtils.Copy(s, fs, buf);
}*/
}
}
}
}
In the example above I use a MemoryStream to store the ZipEntry in memory (for further analysis). You could also store the ZipEntry (if it meets certain criteria) on disk.
Hope, this helps.