Develop Reference

C# - How can I download a zip file from url, unzip it, and read the extracted files, all in memory? [duplicate]

I have files (from 3rd parties) that are being FTP'd to a directory on our server. I download them and process them even 'x' minutes. Works great.
Now, some of the files are .zip files. Which means I can't process them. I need to unzip them first.
FTP has no concept of zip/unzipping - so I'll need to grab the zip file, unzip it, then process it.
Looking at the MSDN zip api, there seems to be no way i can unzip to a memory stream?
So is the only way to do this...
Unzip to a file (what directory? need some -very- temp location ...)
Read the file contents
Delete file.
NOTE: The contents of the file are small - say 4k <-> 1000k.

Zip compression support is built in:
using System.IO;
using System.IO.Compression;
// ^^^ requires a reference to System.IO.Compression.dll
static class Program
{
const string path = ...
static void Main()
{
using(var file = File.OpenRead(path))
using(var zip = new ZipArchive(file, ZipArchiveMode.Read))
{
foreach(var entry in zip.Entries)
{
using(var stream = entry.Open())
{
// do whatever we want with stream
// ...
}
}
}
}
}
Normally you should avoid copying it into another stream - just use it "as is", however, if you absolutely need it in a MemoryStream, you could do:
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0; // rewind
// do something with ms
}

You can use ZipArchiveEntry.Open to get a stream.
This code assumes the zip archive has one text file.
using (FileStream fs = new FileStream(path, FileMode.Open))
using (ZipArchive zip = new ZipArchive(fs) )
{
var entry = zip.Entries.First();
using (StreamReader sr = new StreamReader(entry.Open()))
{
Console.WriteLine(sr.ReadToEnd());
}
}

using (ZipArchive archive = new ZipArchive(webResponse.GetResponseStream()))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Stream s = entry.Open();
var sr = new StreamReader(s);
var myStr = sr.ReadToEnd();
}
}

Looks like here is what you need:
using (var za = ZipFile.OpenRead(path))
{
foreach (var entry in za.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
//your code here
}
}
}

You can use SharpZipLib among a variety of other libraries to achieve this.
You can use the following code example to unzip to a MemoryStream, as shown on their wiki:
using ICSharpCode.SharpZipLib.Zip;
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName) {
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}

Ok so combining all of the above, suppose you want to in a very simple way take a zip file called
"file.zip" and extract it to "C:\temp" folder. (Note: This example was only tested for compress text files) You may need to do some modifications for binary files.
using System.IO;
using System.IO.Compression;
static void Main(string[] args)
{
//Call it like this:
Unzip("file.zip",#"C:\temp");
}
static void Unzip(string sourceZip, string targetPath)
{
using (var z = ZipFile.OpenRead(sourceZip))
{
foreach (var entry in z.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
string uncompressedFile = Path.Combine(targetPath, entry.Name);
File.WriteAllText(uncompressedFile,r.ReadToEnd());
}
}
}
}

Get zip file entries in .NET 2.0

I need to get the number of entries and original file size of each entry in a ZIP file. On .NET 4.5, we can use the C# classes ZipArchive and ZipFile as shown below, but these classes are not available in .NET 2.0. What are my options then?
using (ZipArchive archive = ZipFile.OpenRead(fileName))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Console.WriteLine("[INFO] Zip entry {0} original file size: [{1}]", entry.name, entry.Length);
}
}
Thanks in advance.

Using [ICsharpZipLib] (https://icsharpcode.github.io/SharpZipLib/)
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;
public void ExtractZipFile(string archiveFilenameIn, string password, string outFolder) {
ZipFile zf = null;
try {
FileStream fs = File.OpenRead(archiveFilenameIn);
zf = new ZipFile(fs);
if (!String.IsNullOrEmpty(password)) {
zf.Password = password; // AES encrypted entries are handled automatically
}
foreach (ZipEntry zipEntry in zf) {
if (!zipEntry.IsFile) {
continue; // Ignore directories
}
String entryFileName = zipEntry.Name;
// to remove the folder from the entry:- entryFileName = Path.GetFileName(entryFileName);
// Optionally match entrynames against a selection list here to skip as desired.
// The unpacked length is available in the zipEntry.Size property.
byte[] buffer = new byte[4096]; // 4K is optimum
Stream zipStream = zf.GetInputStream(zipEntry);
// Manipulate the output filename here as desired.
String fullZipToPath = Path.Combine(outFolder, entryFileName);
string directoryName = Path.GetDirectoryName(fullZipToPath);
if (directoryName.Length > 0)
Directory.CreateDirectory(directoryName);
// Unzip file in buffered chunks. This is just as fast as unpacking to a buffer the full size
// of the file, but does not waste memory.
// The "using" will close the stream even if an exception occurs.
using (FileStream streamWriter = File.Create(fullZipToPath)) {
StreamUtils.Copy(zipStream, streamWriter, buffer);
}
}
} finally {
if (zf != null) {
zf.IsStreamOwner = true; // Makes close also shut the underlying stream
zf.Close(); // Ensure we release resources
}
}
}

How can I unzip a file to a .NET memory stream?

I have files (from 3rd parties) that are being FTP'd to a directory on our server. I download them and process them even 'x' minutes. Works great.
Now, some of the files are .zip files. Which means I can't process them. I need to unzip them first.
FTP has no concept of zip/unzipping - so I'll need to grab the zip file, unzip it, then process it.
Looking at the MSDN zip api, there seems to be no way i can unzip to a memory stream?
So is the only way to do this...
Unzip to a file (what directory? need some -very- temp location ...)
Read the file contents
Delete file.
NOTE: The contents of the file are small - say 4k <-> 1000k.

Zip compression support is built in:
using System.IO;
using System.IO.Compression;
// ^^^ requires a reference to System.IO.Compression.dll
static class Program
{
const string path = ...
static void Main()
{
using(var file = File.OpenRead(path))
using(var zip = new ZipArchive(file, ZipArchiveMode.Read))
{
foreach(var entry in zip.Entries)
{
using(var stream = entry.Open())
{
// do whatever we want with stream
// ...
}
}
}
}
}
Normally you should avoid copying it into another stream - just use it "as is", however, if you absolutely need it in a MemoryStream, you could do:
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0; // rewind
// do something with ms
}

You can use ZipArchiveEntry.Open to get a stream.
This code assumes the zip archive has one text file.
using (FileStream fs = new FileStream(path, FileMode.Open))
using (ZipArchive zip = new ZipArchive(fs) )
{
var entry = zip.Entries.First();
using (StreamReader sr = new StreamReader(entry.Open()))
{
Console.WriteLine(sr.ReadToEnd());
}
}

using (ZipArchive archive = new ZipArchive(webResponse.GetResponseStream()))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Stream s = entry.Open();
var sr = new StreamReader(s);
var myStr = sr.ReadToEnd();
}
}

Looks like here is what you need:
using (var za = ZipFile.OpenRead(path))
{
foreach (var entry in za.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
//your code here
}
}
}

You can use SharpZipLib among a variety of other libraries to achieve this.
You can use the following code example to unzip to a MemoryStream, as shown on their wiki:
using ICSharpCode.SharpZipLib.Zip;
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName) {
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}

Ok so combining all of the above, suppose you want to in a very simple way take a zip file called
"file.zip" and extract it to "C:\temp" folder. (Note: This example was only tested for compress text files) You may need to do some modifications for binary files.
using System.IO;
using System.IO.Compression;
static void Main(string[] args)
{
//Call it like this:
Unzip("file.zip",#"C:\temp");
}
static void Unzip(string sourceZip, string targetPath)
{
using (var z = ZipFile.OpenRead(sourceZip))
{
foreach (var entry in z.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
string uncompressedFile = Path.Combine(targetPath, entry.Name);
File.WriteAllText(uncompressedFile,r.ReadToEnd());
}
}
}
}

.Net Zip Up files

Whats the best way to zip up files using C#? Ideally I want to be able to seperate files into a single archive.

You can use DotNetZip to archieve this. It´s free to use in any application.
Here´s some sample code:
try
{
// for easy disposal
using (ZipFile zip = new ZipFile())
{
// add this map file into the "images" directory in the zip archive
zip.AddFile("c:\\images\\personal\\7440-N49th.png", "images");
// add the report into a different directory in the archive
zip.AddFile("c:\\Reports\\2008-Regional-Sales-Report.pdf", "files");
zip.AddFile("ReadMe.txt");
zip.Save("MyZipFile.zip");
}
}
catch (System.Exception ex1)
{
System.Console.Error.WriteLine("exception: " + ex1);
}

This is now built into the framework if you have version 4.5+
Otherwise, use Ionic.
Namespace is System.IO.Packaging.ZIPPackage.
See http://visualstudiomagazine.com/articles/2012/05/21/net-framework-gets-zip.aspx for a story.

Have you looked at SharpZipLib?
I believe you can build zip files with classes in the System.IO.Packaging namespace - but every time I've tried to look into it, I've found it rather confusing...

Take a look at this library:
http://www.icsharpcode.net/OpenSource/SharpZipLib/
It is pretty comprehensive, it deals with many formats, is open-source, and you can use in closed-source commercial applications.
It is very simple to use:
byte[] data1 = new byte[...];
byte[] data2 = new byte[...];
/*...*/
var path = #"c:\test.zip";
var zip = new ZipOutputStream(new FileStream(path, FileMode.Create))
{
IsStreamOwner = true
}
zip.PutNextEntry("File1.txt");
zip.Write(data1, 0, data1.Length);
zip.PutNextEntry("File2.txt");
zip.Write(data2, 0, data2.Length);
zip.Close();
zip.Dispose();

There are a few librarys around - the most popular of which are DotNetZip and SharpZipLib.

Hi i created two methods with the ShapLib library (you can download it here http://www.icsharpcode.net/opensource/sharpziplib/) that would like to share, they are very easy to use just pass source and target path (fullpath including folder/file and extension). Hope it help you!
//ALLYOURNAMESPACESHERE
using ...
//SHARPLIB
using ICSharpCode.SharpZipLib.Zip;
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
public static class FileUtils
{
/// <summary>
///
/// </summary>
/// <param name="sourcePath"></param>
/// <param name="targetPath"></param>
public static void ZipFile(string sourcePath, string targetPath)
{
string tempZipFilePath = targetPath;
using (FileStream tempFileStream = File.Create(tempZipFilePath, 1024))
{
using (ZipOutputStream zipOutput = new ZipOutputStream(tempFileStream))
{
// Zip with highest compression.
zipOutput.SetLevel(9);
DirectoryInfo directory = new DirectoryInfo(sourcePath);
foreach (System.IO.FileInfo file in directory.GetFiles())
{
// Get local path and create stream to it.
String localFilename = file.FullName;
//ignore directories or folders
//ignore Thumbs.db file since this probably will throw an exception
//another process could be using it. e.g: Explorer.exe windows process
if (!file.Name.Contains("Thumbs") && !Directory.Exists(localFilename))
{
using (FileStream fileStream = new FileStream(localFilename, FileMode.Open, FileAccess.Read, FileShare.Read))
{
// Read full stream to in-memory buffer.
byte[] buffer = new byte[fileStream.Length];
fileStream.Read(buffer, 0, buffer.Length);
// Create a new entry for the current file.
ZipEntry entry = new ZipEntry(file.Name);
entry.DateTime = DateTime.Now;
// set Size and the crc, because the information
// about the size and crc should be stored in the header
// if it is not set it is automatically written in the footer.
// (in this case size == crc == -1 in the header)
// Some ZIP programs have problems with zip files that don't store
// the size and crc in the header.
entry.Size = fileStream.Length;
fileStream.Close();
// Update entry and write to zip stream.
zipOutput.PutNextEntry(entry);
zipOutput.Write(buffer, 0, buffer.Length);
// Get rid of the buffer, because this
// is a huge impact on the memory usage.
buffer = null;
}
}
}
// Finalize the zip output.
zipOutput.Finish();
// Flushes the create and close.
zipOutput.Flush();
zipOutput.Close();
}
}
}
public static void unZipFile(string sourcePath, string targetPath)
{
if (!Directory.Exists(targetPath))
Directory.CreateDirectory(targetPath);
using (ZipInputStream s = new ZipInputStream(File.OpenRead(sourcePath)))
{
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
if (theEntry.Name != String.Empty)
{
using (FileStream streamWriter = File.Create(targetPath + "\\" + theEntry.Name))
{
int size = 2048;
byte[] data = new byte[2048];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}
}
}
}
}
}

Unzipping a file error

I am using the SharpZipLib open source .net library from www.icsharpcode.net
My goal is to unzip an xml file and read it into a dataset. However I get the following error reading the file into a dataset: "Data at the root level is invalid. Line 1, position 1."
I believe what is happening is the unzipping code is not releasing the file for the following reasons.
1.) If I unzip the file and exit the application. When I restart the app I CAN read the unzipped file into a dataset.
2.) If I read in the xml file right after writing it out (no zipping) then it works fine.
3.) If I write the dataset to xml, zip it up, unzip it, then attempt to read it back in I get the exception.
The code below is pretty straight forward. UnZipFile will return the name of the file just unzipped. Right below this call is the call to read it into a dataset. The variable fileToRead is the full path to the newly unzipped xml file.
string fileToRead = UnZipFile(filepath, DOViewerUploadStoreArea);
ds.ReadXml(fileToRead )
private string UnZipFile(string file, string dirToUnzipTo)
{
string unzippedfile = "";
try
{
ZipInputStream s = new ZipInputStream(File.OpenRead(file));
ZipEntry myEntry;
string tmpEntry = String.Empty;
while ((myEntry = s.GetNextEntry()) != null)
{
string directoryName = dirToUnzipTo;
string fileName = Path.GetFileName(myEntry.Name);
string fileWDir = directoryName + fileName;
unzippedfile = fileWDir;
FileStream streamWriter = File.Create(fileWDir);
int size = 4096;
byte[] data = new byte[4096];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0) { streamWriter.Write(data, 0, size); }
else { break; }
}
streamWriter.Close();
}
s.Close();
}
catch (Exception ex)
{
LogStatus.WriteErrorLog(ex, "ERROR", "DOViewer.UnZipFile");
}
return (unzippedfile);
}

Well, what does the final file look like? (compared to the original). You don't show the zipping code, which might be part of the puzzle, especially as you are partially swallowing the exception.
I would also try ensuring everything IDisposable is Dispose()d, ideally via using; also - in case the problem is with path construction, use Path.Combine. And note that if myEntry.Name contains sub-directories, you will need to create them manually.
Here's what I have - it works for unzipping ICSharpCode.SharpZipLib.dll:
private string UnZipFile(string file, string dirToUnzipTo)
{
string unzippedfile = "";
try
{
using(Stream inStream = File.OpenRead(file))
using (ZipInputStream s = new ZipInputStream(inStream))
{
ZipEntry myEntry;
byte[] data = new byte[4096];
while ((myEntry = s.GetNextEntry()) != null)
{
string fileWDir = Path.Combine(dirToUnzipTo, myEntry.Name);
string dir = Path.GetDirectoryName(fileWDir);
// note only supports a single level of sub-directories...
if (!Directory.Exists(dir)) Directory.CreateDirectory(dir);
unzippedfile = fileWDir; // note; returns last file if multiple
using (FileStream outStream = File.Create(fileWDir))
{
int size;
while ((size = s.Read(data, 0, data.Length)) > 0)
{
outStream.Write(data, 0, size);
}
outStream.Close();
}
}
s.Close();
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
return (unzippedfile);
}
It could also be that the problem is either in the code that writes the zip, or the code that reads the generated file.

I compared the original with the final using TextPad and they are identical.
Also I rewrote the code to take advantage of the using. Here is the code.
My issue seems to be centered around file locking or something. If I unzip the file quit the application then start it up it will read find.
private string UnZipFile(string file, string dirToUnzipTo)
{
string unzippedfile = "";
try
{
using (ZipInputStream s = new ZipInputStream(File.OpenRead(file)))
{
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
string directoryName = dirToUnzipTo;
string fileName = Path.GetFileName(theEntry.Name);
string fileWDir = directoryName + fileName;
unzippedfile = fileWDir;
if (fileName != String.Empty)
{
using (FileStream streamWriter = File.Create(fileWDir))
{
int size = 2048;
byte[] data = new byte[2048];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}
}
}
}
}
catch (Exception ex)
{
LogStatus.WriteErrorLog(ex, "ERROR", "DOViewer.UnZipFile");
}
return (unzippedfile);
}

This is a lot simpler to do with DotNetZip.
using (ZipFile zip = ZipFile.Read(ExistingZipFile))
{
zip.ExtractAll(TargetDirectory);
}
If you want to decide on which files to extract ....
using (ZipFile zip = ZipFile.Read(ExistingZipFile))
{
foreach (ZipEntry e in zip)
{
if (wantThisFile(e.FileName)) e.Extract(TargetDirectory);
}
}
If you would like to overwrite existing files during extraction:
using (ZipFile zip = ZipFile.Read(ExistingZipFile))
{
zip.ExtractAll(TargetDirectory, ExtractExistingFileAction.OverwriteSilently);
}
Or, to extract password-protected entries:
using (ZipFile zip = ZipFile.Read(ExistingZipFile))
{
zip.Password = "Shhhh, Very Secret!";
zip.ExtractAll(TargetDirectory, ExtractExistingFileAction.OverwriteSilently);
}