I am trying to programatically unzip a zipped file.
I have tried using the System.IO.Compression.GZipStream class in .NET, but when my app runs (actually a unit test) I get this exception:
System.IO.InvalidDataException: The magic number in GZip header is not correct. Make sure you are passing in a GZip stream..
I now realize that a .zip file is not the same as a .gz file, and that GZip is not the same as Zip.
However, since I'm able to extract the file by manually double clicking the zipped file and then clicking the "Extract all files"-button, I think there should be a way of doing that in code as well.
Therefore I've tried to use Process.Start() with the path to the zipped file as input. This causes my app to open a Window showing the contents in the zipped file. That's all fine, but the app will be installed on a server with none around to click the "Extract all files"-button.
So, how do I get my app to extract the files in the zipped files?
Or is there another way to do it? I prefer doing it in code, without downloading any third party libraries or apps; the security department ain't too fancy about that...
With .NET 4.5 you can now unzip files using the .NET framework:
using System;
using System.IO;
namespace ConsoleApplication
{
class Program
{
static void Main(string[] args)
{
string startPath = #"c:\example\start";
string zipPath = #"c:\example\result.zip";
string extractPath = #"c:\example\extract";
System.IO.Compression.ZipFile.CreateFromDirectory(startPath, zipPath);
System.IO.Compression.ZipFile.ExtractToDirectory(zipPath, extractPath);
}
}
}
The above code was taken directly from Microsoft's documentation: http://msdn.microsoft.com/en-us/library/ms404280(v=vs.110).aspx
ZipFile is contained in the assembly System.IO.Compression.FileSystem. (Thanks nateirvin...see comment below). You need to add a DLL reference to the framework assembly System.IO.Compression.FileSystem.dll
For .Net 4.5+
It is not always desired to write the uncompressed file to disk. As an ASP.Net developer, I would have to fiddle with permissions to grant rights for my application to write to the filesystem. By working with streams in memory, I can sidestep all that and read the files directly:
using (ZipArchive archive = new ZipArchive(postedZipStream))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
var stream = entry.Open();
//Do awesome stream stuff!!
}
}
Alternatively, you can still write the decompressed file out to disk by calling ExtractToFile():
using (ZipArchive archive = ZipFile.OpenRead(pathToZip))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
entry.ExtractToFile(Path.Combine(destination, entry.FullName));
}
}
To use the ZipArchive class, you will need to add a reference to the System.IO.Compression namespace and to System.IO.Compression.FileSystem.
We have used SharpZipLib successfully on many projects. I know it's a third party tool, but source code is included and could provide some insight if you chose to reinvent the wheel here.
Free, and no external DLL files. Everything is in one CS file. One download is just the CS file, another download is a very easy to understand example. Just tried it today and I can't believe how simple the setup was. It worked on first try, no errors, no nothing.
https://github.com/jaime-olivares/zipstorer
Use the DotNetZip library at http://www.codeplex.com/DotNetZip
class library and toolset for manipulating zip files. Use VB, C# or any .NET language to easily create, extract, or update zip files...
DotNetZip works on PCs with the full .NET Framework, and also runs on mobile devices that use the .NET Compact Framework. Create and read zip files in VB, C#, or any .NET language, or any scripting environment...
If all you want is a better DeflateStream or GZipStream class to replace the one that is built-into the .NET BCL, DotNetZip has that, too. DotNetZip's DeflateStream and GZipStream are available in a standalone assembly, based on a .NET port of Zlib. These streams support compression levels and deliver much better performance than the built-in classes. There is also a ZlibStream to complete the set (RFC 1950, 1951, 1952)...
String ZipPath = #"c:\my\data.zip";
String extractPath = #"d:\\myunzips";
ZipFile.ExtractToDirectory(ZipPath, extractPath);
To use the ZipFile class, you must add a reference to the System.IO.Compression.FileSystem assembly in your project
This will do it System.IO.Compression.ZipFile.ExtractToDirectory(ZipName, ExtractToPath)
Standard zip files normally use the deflate algorithm.
To extract files without using third party libraries use DeflateStream. You'll need a bit more information about the zip file archive format as Microsoft only provides the compression algorithm.
You may also try using zipfldr.dll. It is Microsoft's compression library (compressed folders from the Send to menu). It appears to be a com library but it's undocumented. You may be able to get it working for you through experimentation.
I use this to either zip or unzip multiple files. The Regex stuff is not required, but I use it to change the date stamp and remove unwanted underscores. I use the empty string in the Compress >> zipPath string to prefix something to all files if required. Also, I usually comment out either Compress() or Decompress() based on what I am doing.
using System;
using System.IO.Compression;
using System.IO;
using System.Text.RegularExpressions;
namespace ZipAndUnzip
{
class Program
{
static void Main(string[] args)
{
var directoryPath = new DirectoryInfo(#"C:\your_path\");
Compress(directoryPath);
Decompress(directoryPath);
}
public static void Compress(DirectoryInfo directoryPath)
{
foreach (DirectoryInfo directory in directoryPath.GetDirectories())
{
var path = directoryPath.FullName;
var newArchiveName = Regex.Replace(directory.Name, "[0-9]{8}", "20130913");
newArchiveName = Regex.Replace(newArchiveName, "[_]+", "_");
string startPath = path + directory.Name;
string zipPath = path + "" + newArchiveName + ".zip";
ZipFile.CreateFromDirectory(startPath, zipPath);
}
}
public static void Decompress(DirectoryInfo directoryPath)
{
foreach (FileInfo file in directoryPath.GetFiles())
{
var path = directoryPath.FullName;
string zipPath = path + file.Name;
string extractPath = Regex.Replace(path + file.Name, ".zip", "");
ZipFile.ExtractToDirectory(zipPath, extractPath);
}
}
}
}
You can do it all within .NET 3.5 using DeflateStream. The thing lacking in .NET 3.5 is the ability to process the file header sections that are used to organize the zipped files. PKWare has published this information, which you can use to process the zip file after you create the structures that are used. It is not particularly onerous, and it a good practice in tool building without using 3rd party code.
It isn't a one line answer, but it is completely doable if you are willing and able to take the time yourself. I wrote a class to do this in a couple of hours and what I got from that is the ability to zip and unzip files using .NET 3.5 only.
From here :
Compressed GZipStream objects written
to a file with an extension of .gz can
be decompressed using many common
compression tools; however, this class
does not inherently provide
functionality for adding files to or
extracting files from .zip archives.
I found out about this one (Unzip package on NuGet) today, since I ran into a hard bug in DotNetZip, and I realized there hasn't been really that much work done on DotNetZip for the last two years.
The Unzip package is lean, and it did the job for me - it didn't have the bug that DotNetZip had. Also, it was a reasonably small file, relying upon the Microsoft BCL for the actual decompression. I could easily make adjustments which I needed (to be able to keep track of the progress while decompressing). I recommend it.
From Embed Ressources:
using (Stream _pluginZipResourceStream = Assembly.GetExecutingAssembly().GetManifestResourceStream(programName + "." + "filename.zip"))
{
using (ZipArchive zip = new ZipArchive(_pluginZipResourceStream))
{
zip.ExtractToDirectory(Application.StartupPath);
}
}
Until now, I was using cmd processes in order to extract an .iso file, copy it into a temporary path from server and extracted on a usb stick. Recently I've found that this is working perfectly with .iso's that are less than 10Gb. For a iso like 29Gb this method gets stuck somehow.
public void ExtractArchive()
{
try
{
try
{
Directory.Delete(copyISOLocation.OutputPath, true);
}
catch (Exception e) when (e is IOException || e is UnauthorizedAccessException)
{
}
Process cmd = new Process();
cmd.StartInfo.FileName = "cmd.exe";
cmd.StartInfo.RedirectStandardInput = true;
cmd.StartInfo.RedirectStandardOutput = true;
cmd.StartInfo.CreateNoWindow = true;
cmd.StartInfo.UseShellExecute = false;
cmd.StartInfo.WindowStyle = ProcessWindowStyle.Normal;
//stackoverflow
cmd.StartInfo.Arguments = "-R";
cmd.Disposed += (sender, args) => {
Console.WriteLine("CMD Process disposed");
};
cmd.Exited += (sender, args) => {
Console.WriteLine("CMD Process exited");
};
cmd.ErrorDataReceived += (sender, args) => {
Console.WriteLine("CMD Process error data received");
Console.WriteLine(args.Data);
};
cmd.OutputDataReceived += (sender, args) => {
Console.WriteLine("CMD Process Output data received");
Console.WriteLine(args.Data);
};
//stackoverflow
cmd.Start();
cmd.StandardInput.WriteLine("C:");
//Console.WriteLine(cmd.StandardOutput.Read());
cmd.StandardInput.Flush();
cmd.StandardInput.WriteLine("cd C:\\\"Program Files (x86)\"\\7-Zip\\");
//Console.WriteLine(cmd.StandardOutput.ReadToEnd());
cmd.StandardInput.Flush();
cmd.StandardInput.WriteLine(string.Format("7z.exe x -o{0} {1}", copyISOLocation.OutputPath, copyISOLocation.TempIsoPath));
//Console.WriteLine(cmd.StandardOutput.ReadToEnd());
cmd.StandardInput.Flush();
cmd.StandardInput.Close();
cmd.WaitForExit();
Console.WriteLine(cmd.StandardOutput.ReadToEnd());
Console.WriteLine(cmd.StandardError.ReadToEnd());
you can use Info-unzip command line cod.you only need to download unzip.exe from Info-unzip official website.
internal static void Unzip(string sorcefile)
{
try
{
AFolderFiles.AFolderFilesDelete.DeleteFolder(TempBackupFolder); // delete old folder
AFolderFiles.AFolderFilesCreate.CreateIfNotExist(TempBackupFolder); // delete old folder
//need to Command command also to export attributes to a excel file
System.Diagnostics.Process process = new System.Diagnostics.Process();
System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
startInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden; // window type
startInfo.FileName = UnzipExe;
startInfo.Arguments = sorcefile + " -d " + TempBackupFolder;
process.StartInfo = startInfo;
process.Start();
//string result = process.StandardOutput.ReadToEnd();
process.WaitForExit();
process.Dispose();
process.Close();
}
catch (Exception ex){ throw ex; }
}
I have a couple of tools that are initialized as a process by my .NET application. Some of them are from third parties. The list of tools to start is configurable. I want to protect the system against executing (maliciously) modified tools.
Assumption and possible attack vectors
I assume my application is protected by the operating system, including its configuration file. The folders containing the tools to be started are not though. Meaning an attacker could theoretically replace any of the tools with a different program bearing the same name.
Possible solution
My current concept would be to add SHA256 hashes of each tool to the config. This should prevent them from being replaced by manipulated applications, assuming my application and the configuration file containing the list is protected.
Implementation example
Config file example:
<FileHashes>
<Hashes>
<add Filename="the-tool.exe" Hash="87BC21C157F7B3E4..." />
</Hashes>
</FileHashes>
Application example:
public void Execute(FileInfo fileToLoad)
{
var hashDictionary = LoadHashDictionary(); //load hash from config
if (!hashDictionary.TryGetValue(fileToLoad.Name, out string configFileHash))
{
throw new Exception("Unknown file");
}
var fileHash = GetFileHash(fileToLoad);
if (!configFileHash.ToUpperInvariant().Equals(fileHash))
{
throw new Exception("File is manipulated");
}
Process.Start(new ProcessStartInfo()
{
FileName = fileToLoad.FullName
});
}
public string GetFileHash(FileInfo fileToLoad)
{
using (var fileStream = new FileStream(fileToLoad.FullName, FileMode.Open))
{
using (var shaHash = SHA256.Create())
{
fileStream.Position = 0;
byte[] hasValue = shaHash.ComputeHash(fileStream);
return BitConverter.ToString(hasValue)
.Replace("-", string.Empty)
.ToUpperInvariant();
}
}
}
Questions
Are there better ways to protect against manipulated files on application level?
Assuming the configuration file isn't safe, as it is also placed in a folder that can theoretically be accessed by an attacker. Where could I place the application whitelist and the hashes?
As you said. Confg file isn't safe either. Malicious application can modify your tools + config files all together to mask itself.
Whats an easy way to prevent this? Code signing.
Simply sign all your tools and do a certificate verification before you start your tool from main app.
I have a project that uses an Access DB file for reference tables. This is to be used at work, but I am developing it at home. Up until now, I've simply run the debugger in VS2010, then copied the relevant class files, exe, etc from the /bin folder to a flash drive, and it's worked fine. But with the DB added in, it suddenly crashes on launch.
I know the problem is the file location of the DB file. Originally the Build Action of the DB was sent to Content. I have changed it to Embedded Resource, which as far as I understand means it will be part of the exe file now.
Am I correct in this? If not, what option do I need to select to have the DB become just a compiled part of the exe, or one of the other dll's?
If the db file is embedded, you can't access it to add/removes rows etc.
Why did you change the build action to Embedded Resource ? It'll be better to put as Content, so the db is a separate file than the exe (but still in the same directory), and then build the path to the db file (i.e. using Application.StartupPath).
Anyway, if you want to set it as Embedded you'll need to extract the db at runtime and store it somewhere before using it.
Here is a method that can extract a file from the embedded resources (of course you'll need to change the filename, or pass it as argument):
private void ExtractFromAssembly()
{
string strPath = Application.LocalUserAppDataPath + "\\MyFile.db";
if (File.Exists(strPath)) return; // already exist, don't overwrite
Assembly assembly = Assembly.GetExecutingAssembly();
//In the next line you should provide NameSpace.FileName.Extension that you have embedded
var input = assembly.GetManifestResourceStream("MyFile.db");
var output = File.Open(strPath, FileMode.CreateNew);
CopyStream(input, output);
input.Dispose();
output.Dispose();
System.Diagnostics.Process.Start(strPath);
}
private void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[32768];
while (true)
{
int read = input.Read(buffer, 0, buffer.Length);
if (read <= 0)
return;
output.Write(buffer, 0, read);
}
}
The file will be copied in the local application path, in the user directory. It'll be done the first time the app is started, because otherwise the db file will be overwritten each time the application start (overwritten with the clean db package in the exe)
I've got a windows program using C# that is working with log files. Some of these different log files come in gzipped (for instance test.log.gz). I've got code using SharpZipLib to unzip those log files and it works really well.
public static void unZip(string gzipFilePath, string targetDir)
{
byte[] dataBuffer = new byte[4096];
using (System.IO.Stream fs = new FileStream(gzipFilePath, FileMode.Open, FileAccess.Read))
{
using (GZipInputStream gzipStream = new GZipInputStream(fs))
{
string fnOut = Path.Combine(targetDir, Path.GetFileNameWithoutExtension(gzipFilePath));
using (FileStream fsOut = File.Create(fnOut))
{
StreamUtils.Copy(gzipStream, fsOut, dataBuffer);
}
}
}
}
From my research, it would seem that gzip files are typically one file, so it's always for instance, test.htm.gz. So I would create a file named test.htm and put the uncompressed information into test.htm, which happens in this part of the code:
using (GZipInputStream gzipStream = new GZipInputStream(fs))
{
string fnOut = Path.Combine(targetDir, Path.GetFileNameWithoutExtension(gzipFilePath));
using (FileStream fsOut = File.Create(fnOut))
{
StreamUtils.Copy(gzipStream, fsOut, dataBuffer);
}
}
This is all well and good but the problem I'm having is I've been given a log file, for example again, test.log.gz that has directories zipped into it.
When I use the 7-Zip gui to unzip the file, the log file I need is five directories deep in folders. So after unzipping with 7-zip, it outputs:
folder1 -> folder2 -> folder3 -> folder4 -> folder5 -> test.log
Trying to use the method provided from SharpLib only gives me a small subset of the data of the file in test.log.
I haven't been able to find any code or issues dealing with gzipped files containing folders and from what I can tell, you're not supposed to do that. It should be in a .tar and then gzipped.
Any one have any idea of what I could do with this .gz file?
First Maybe try using another lib here are a few
http://dotnetzip.codeplex.com/
http://www.icsharpcode.net/OpenSource/SharpZipLib/
There is also a built in GZ lib built into .net see
Unzipping a .gz file using C#
There is still just one file in there, so there isn't any violation of the gzip format. gzip permits an entire path name to be stored with the file, so that path may simply be ghostcache/ic_split_files/CBN/00-christmas/test.log and 7-Zip is faithfully recreating that path. You should be able to see this in the gzip header, starting about ten bytes in.
The fact that you are getting back only a subset of the log may or may not be related to the pathname in the gzip file.
Please provide a hex dump of the first 64 bytes of the .gz file that worked and the the .gz file that didn't.
I use to store document/file in byte[] in database, and I want user can view/run that file from my application.
You need to know the file extension for the file you're writing, so the OS can run the default program based on the extension. The code would be something like this:
byte[] bytes = GetYourBytesFromDataBase();
string extension = GetYourFileExtension(); //.doc for example
string path = Path.GetTempFileName() + extension;
try
{
using(BinaryWriter writer = new BinaryWriter(File.Open(path, FileMode.Create)))
{
writer.Write(yourBytes);
}
// open it with default application based in the
// file extension
Process p = System.Diagnostics.Process.Start(path);
p.Wait();
}
finally
{
//clean the tmp file
File.Delete(path);
}
You will need to store the file extension in the database too. If you don't have the file extension the problem becomes very difficult as you cannot rely on the operating system to work out which program to launch to handle the file.
You can use the following pattern:
Load data from database and save to file using the original file extension.
Start a new System.Diagnostics.Process that points to the saved file path.
As you have saved the file with the original file extension, the OS will look for a program that is registered for the extension to open the file.
As chibacity and Daniel suggest, storing the file extension in the db, and agreed -- storing the file extension, or at least some indicator that tells you the file type, is a good idea.
If these files are of a format of your own creation then you might also want to store information about which version of the file format the data is stored in. During development file formats are prone to changing, and if you don't remember which version you used to store the data then you have a hard job recovering the information.
The same problems are faced in object persistence generally.