I have a compressed file (.osz) stored on an S3 Bucket which contains a .os file which I need to read the contents of.
I need to be able to extract/decompress and read the contents of the compressed file without downloading the file directly to the PC due to security reasons.
I am able to retrieve the compressed file (.osz) using the URL address of it's S3 Bucket path. Then using ZipArchive I am able to access the files contained within the compressed file. I am then able to extract the file I require by using the 'ExtractToFile' function as seen below. However, this function extracts the file (.os) and saves it locally in the path specified.
WebClient client = new WebClient();
byte[] bytes = client.DownloadData(OSZFilepath); // Read the .osz file contents into a byte array
using (MemoryStream zipStream = new MemoryStream(bytes))
{
// Create the zip containing the file from the stream
ZipArchive zip = new ZipArchive(zipStream);
// extract the compressed file and download the .os file contained within
var fileName = Guid.NewGuid().ToString() + ".OS";
var baseDirectory = Environment.ExpandEnvironmentVariables(System.Web.Configuration.WebConfigurationManager.AppSettings["DataStorage"].ToString());
zip.Entries[0].ExtractToFile(Path.Combine(baseDirectory, "ProjectFiles", fileName));
Although this extracted file can be successfully read and imported by my program, I cannot use this method as I am not able to save the file onto the user's computer due to security restrictions and program requirements.
Therefore I need to programmatically extract/decompress the file I need in order to read the contents and programmatically import it into my program.
I have tried to use the following code to do this:
ZipArchiveEntry entry = zip.GetEntry(zip.Entries[0].Name);
Stream stream = entry.Open();
StreamReader reader = new StreamReader(stream);
string contents = reader.ReadToEnd();
However the resulting contents throws up an error when I try to import it, indicating that the contents is different to the contents of the file that gets saved using 'ExtractToFile'.
This is confirmed when I save this contents as a seperate file and compare it to the file saved using 'ExtractToFile'. The 'ExtractToFile' file is bigger than the latter.
So my question is: is there another way to successfully decompress/extract a compressed file and obtain the contents without using the 'ExtractToFile' method and having to save the extracted file somewhere?
Thanks for your help.
Related
I am trying to convert an entire azure blob storage folder and its contents to a zip file .Inside this folder ,I have different types of files eg, .txt,.mp3,.zip files .But once the folder is converted to zip file I noticed that all the .zip file types got corrupted,.How can I prevent my zip files from corrupted. I am using Ionic.Zip library to generate zip files
Here is the code I am using .Here I am able to generate and download the zip file successfully with all other filetypes except the inner zip files.
var allFiles = directory.ListBlobs(new BlobRequestOptions { UseFlatBlobListing = true }).Where(x => x.GetType() == typeof(CloudBlockBlob)).Cast<CloudBlob>();
string xyzblob = directory.Uri.ToString().TrimEnd('/');
var dBlob = blobClient.GetBlobReference(xyzblob);
byte[] fileBytes = null;
fileBytes = dBlob.DownloadByteArray();
foreach (var file in allFiles)
{
using (var fileStream = new MemoryStream(fileBytes))
{
var entryName = file.Uri.ToString().Replace(directory.Uri.ToString(), "");
zipOutputStream.PutNextEntry(entryName);
fileStream.Seek(0, SeekOrigin.Begin);
int count = fileStream.Read(fileBytes, 0, fileBytes.Length);
while (count > 0)
{
zipOutputStream.Write(fileBytes, 0, count);
count = fileStream.Read(fileBytes, 0, fileBytes.Length);
if (!Response.IsClientConnected)
{
break;
}
Response.Flush();
}
fileStream.Close();
}
}
zipOutputStream.Close();
More details
I am downloading a folder ,."myFolder" and its contents from azure blob as a zip file eg, myfolders.zip.
Here is how the file structure inside "myFolder" /azure blob
MyFolder/mymusic/ test.mp3
MyFolder/mytext/ newtext.txt
MyFolder/MyZipfiles/ myzip.zip
My code I posted above will generate a zip all the contents of the folder to create "MyFolder.zip" and will download automatically .Now if you unzip "MyFolder.zip" file , due to some reason , the myzip.zip is getting corrupted.If I try to open myzip.zip file ,its showing a message "windows cannot open the folder ,the compressed zipped folder "myzip.zip" is invalid"
Please help me find a solution so that the .zip files wont get corrupted
I tried to download to stream ,but same results.,The inner zip files are getting corrupted.all other file types are in good shape.
zipOutputStream.PutNextEntry(entryName);
destBlob.DownloadToStream(zipOutputStream);
I am assuming you already tried downloading one of those zip files and opening it, right?
If that is the case, one thing I would suggest is to eliminate the intermediate fileBytes array completely. Using fileBytes as the buffer to fileStream and then reading from fileStream to fileBytes might be the culprit. On the other hand, you start from offset 0 and write to the beginning of fileBytes anyway, so it might be working just fine.
In any case, a more efficient solution is; you can call PutNextEntry and then call the blob object's DownloadToStream method by passing in the zip stream itself. That would simply copy the entire blob directly into the zip stream without having to manage an intermediate buffer.
When it starts to pick the .zip file ,I added BlobReference to .zip file and this resolved the issue
dBlob = blobClient.GetBlobReference(entryName.EndsWith(".zip") ? file.Uri.ToString() : xyzblob);
zipOutputStream.PutNextEntry(entryName);
dBlob.DownloadToStream(zipOutputStream);
I want to extract a exe file. The exe file contain some files and folders. When I try to extract the file using winrar it gets extracted but when I am trying to extract the exe file using some examples I am getting this error:
The magic number in GZip header is not correct. Make sure you are passing in a GZip stream.
I have used some samples and googled a lot for my problem but didn't get my answer, and I have used some libraries also.
I used this code but same error:
public static void Decompress(FileInfo fi)
{
// Get the stream of the source file.
using (FileStream inFile = fi.OpenRead())
{
// Get original file extension, for example
// "doc" from report.doc.gz.
string curFile = fi.FullName;
string origName = curFile.Remove(curFile.Length -
fi.Extension.Length);
//Create the decompressed file.
using (FileStream outFile = File.Create(origName))
{
using (GZipStream Decompress = new GZipStream(inFile,
CompressionMode.Decompress))
{
// Copy the decompression stream
// into the output file.
Decompress.CopyTo(outFile);
Console.WriteLine("Decompressed: {0}", fi.Name);
}
}
}
}
That's because the .exe file is a self-extracting archive...
You should give DotNetZip a try. From the project's FAQ:
Does this library read self-extracting zip files?
Yes. DotNetZip can read self-extracting archives (SFX) generated by WinZip, and WinZip
can read SFX files generated by DotNetZip.
You can install it from Nuget easily.
I've got a windows program using C# that is working with log files. Some of these different log files come in gzipped (for instance test.log.gz). I've got code using SharpZipLib to unzip those log files and it works really well.
public static void unZip(string gzipFilePath, string targetDir)
{
byte[] dataBuffer = new byte[4096];
using (System.IO.Stream fs = new FileStream(gzipFilePath, FileMode.Open, FileAccess.Read))
{
using (GZipInputStream gzipStream = new GZipInputStream(fs))
{
string fnOut = Path.Combine(targetDir, Path.GetFileNameWithoutExtension(gzipFilePath));
using (FileStream fsOut = File.Create(fnOut))
{
StreamUtils.Copy(gzipStream, fsOut, dataBuffer);
}
}
}
}
From my research, it would seem that gzip files are typically one file, so it's always for instance, test.htm.gz. So I would create a file named test.htm and put the uncompressed information into test.htm, which happens in this part of the code:
using (GZipInputStream gzipStream = new GZipInputStream(fs))
{
string fnOut = Path.Combine(targetDir, Path.GetFileNameWithoutExtension(gzipFilePath));
using (FileStream fsOut = File.Create(fnOut))
{
StreamUtils.Copy(gzipStream, fsOut, dataBuffer);
}
}
This is all well and good but the problem I'm having is I've been given a log file, for example again, test.log.gz that has directories zipped into it.
When I use the 7-Zip gui to unzip the file, the log file I need is five directories deep in folders. So after unzipping with 7-zip, it outputs:
folder1 -> folder2 -> folder3 -> folder4 -> folder5 -> test.log
Trying to use the method provided from SharpLib only gives me a small subset of the data of the file in test.log.
I haven't been able to find any code or issues dealing with gzipped files containing folders and from what I can tell, you're not supposed to do that. It should be in a .tar and then gzipped.
Any one have any idea of what I could do with this .gz file?
First Maybe try using another lib here are a few
http://dotnetzip.codeplex.com/
http://www.icsharpcode.net/OpenSource/SharpZipLib/
There is also a built in GZ lib built into .net see
Unzipping a .gz file using C#
There is still just one file in there, so there isn't any violation of the gzip format. gzip permits an entire path name to be stored with the file, so that path may simply be ghostcache/ic_split_files/CBN/00-christmas/test.log and 7-Zip is faithfully recreating that path. You should be able to see this in the gzip header, starting about ten bytes in.
The fact that you are getting back only a subset of the log may or may not be related to the pathname in the gzip file.
Please provide a hex dump of the first 64 bytes of the .gz file that worked and the the .gz file that didn't.
I have stored my zip file in blob storage . I already read archive from blob to stream
.Code is as follows
string blobUrl = http://127.0.0.1:10000/devstoreaccount1/usercontrols/ucProfileViewSMSIS.zip";
string containerName = "usercontrols";
Storage.Blob blobHandler = new Storage.Blob();
Stream blobstream = blobHandler.GetBlob(blobUrl, containerName);
I have three files in my archive . I want to write write those 3 files to my local folder .
How do I do this ?
You will need an unzip library like DotNetZip to unzip the files. Under the examples section, there is a method to unzip directly from a stream:
Input from a stream. This example reads in zip archive content from an
input stream, then extracts the content for one entry to a filesysten
file. In this example, the filename "NameOfEntryInArchive.doc", refers
only to the name of the entry within the zip archive. This name is
used as the index in the string indexer on the ZipFile object. The
return value is a ZipEntry. The ZipEntry.Extract() method is then
called, which extracts the named entry to a filesystem file, using the
current working directory as the base. A file by that name is created
in the filesystem.
using (ZipFile zip = ZipFile.Read(InputStream))
{
ZipEntry entry = zip["NameOfEntryInArchive.doc"];
entry.Extract(); // create filesystem file here.
}
i am trying to use GZipStream to create a gz file using c#.
my problem is that i have a list that contains strings. and i need to create a password protected zip file, and put in it a text file containing the strings.
i don't want to create the textfile, then zip it, and then delete the textfile. i want to directly create a password protected zip file that contains the text file.
any help?
EDIT: i am done with the zipping stuff. now i need to set a pass for the created zip file. any help?
You should consider using SharpZipLib. It's an open source .net compression library. It includes examples on how to create either a .gz or a .zip file. Note that you can write directly to the .zip file. You don't need to create an intermediate file on disk first.
Edit: (in response to your edit) SharpZipLib supports zip passwords too.
Just create a StreamWriter wrapping your GZipStream, and write text to it.
GZipStream can be used to create a .gz file, but this is not the same as a .zip file.
For creating password-protected zip files, I think you need to go to a third-party library.
Here's how to do it using DotNetZip...
var sb = new System.Text.StringBuilder();
sb.Append("This is the text file...");
foreach (var item in listOfStrings)
sb.Append(item);
// sb now contains all the content that will be placed into
// the text file entry inside the zip.
using (var zip = new Ionic.Zip.ZipFile())
{
// set the password on the zip (implicitly enables encryption)
zip.Password = "Whatever.You.Like!!";
// optional: select strong encryption
zip.Encryption = Ionic.Zip.EncryptionAlgorithm.WinZipAes256;
// add an entry to the zip, specify a name, specify string content
zip.AddEntry("NameOfFile.txt", sb.ToString());
// save the file
zip.Save("MyFile.zip");
}