I have a function that requires a Filestream as input.
I want to hand several Files to that function which I get from uploaded zip-Files.
Is it possible to create the Filestream without extracting the file to a temporary folder?
I imagin something like this:
string path = #"C:\somepathtomyzip";
string filepath = "nameofimagefile"
using (ZipArchive archive = ZipFile.OpenRead(path))
{
ZipArchiveEntry entry = archive.GetEntry(file_path);
//generate Filestream from entry
myFunction(filestreamIneed);
}
You can use ZipArchiveEntry.Open() and copy the output from the returned Stream instance to a FileStream instance:
using (ZipArchive archive = ZipFile.OpenRead(path))
{
ZipArchiveEntry entry = archive.GetEntry(file_path);
var memoryStream = return entry.Open();
using (var fileStream = new FileStream(fileName, FileMode.CreateNew, FileAccess.ReadWrite))
{
memoryStream.CopyTo(fileStream); // fileStream is not populated
}
}
Related
The following code creates a zip file from S3 by pulling them into memory and write the final product to a file on disk. However, it is observer it corrupted few file (out of thousands) while creating the zip. I've checked, there is nothing wrong with files which got corrupted during the process, because same file(s) get zipped properly by other means. Any suggestions to fine tune the code?
Code:
public static async Task S3ToZip(List<string> pdfBatch, string zipPath, IAmazonS3 s3Client)
{
FileStream fileStream = new FileStream(zipPath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);
using (ZipArchive archive = new ZipArchive(fileStream, ZipArchiveMode.Update, true))
{
foreach (var file in pdfBatch)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = "sample-bucket",
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
ZipArchiveEntry zipFileEntry = archive.CreateEntry(file.Split('/')[^1]);
using Stream zipEntryStream = zipFileEntry.Open();
await responseStream.CopyToAsync(zipEntryStream);
zipEntryStream.Seek(0, SeekOrigin.Begin);
zipEntryStream.CopyTo(fileStream);
}
archive.Dispose();
fileStream.Close();
}
}
Don't call Dispose() or Close() explicitly, let using do all the job. And you don't need to write anything to fileStream writing to ZipArchiveEntrystream does it under the hood. You also need to use FileMode.Create to guarantee that your file is always truncated before writing to it. Also as you only creating archive not updating it, you should use ZipArchiveMode.Create to enable memory efficient streaming (thanks to #canton7 for some deep diving in details of zip archive format).
public static async Task S3ToZip(List<string> pdfBatch, string zipPath, IAmazonS3 s3Client)
{
using FileStream fileStream = new FileStream(zipPath, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
using ZipArchive archive = new ZipArchive(fileStream, ZipArchiveMode.Create, true);
foreach (var file in pdfBatch)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = "sample-bucket",
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
ZipArchiveEntry zipFileEntry = archive.CreateEntry(file.Split('/')[^1]);
using Stream zipEntryStream = zipFileEntry.Open();
await responseStream.CopyToAsync(zipEntryStream);
}
}
I am using Zip Archive to create a zip folder with various files and subfolders and returning it as a memory stream like so.
public MemoryStream CreateAZipFolder(){
var stMarged = new System.IO.MemoryStream();
stMarged.Position = 0;
using (MemoryStream zipStream = new MemoryStream())
{
using (ZipArchive zip = new ZipArchive(zipStream, ZipArchiveMode.Create, true))
{
string[] fileEntries = Directory.GetFiles(#"C:\Applications\folder");
foreach (var fileName in fileEntries)
{
zip.CreateEntryFromFile(fileName, "Applications/folder/" + Path.GetFileName(fileName),
CompressionLevel.Optimal);
}
ZipArchiveEntry batchEntry = zip.CreateEntry("mybatchFile.bat");
using (StreamWriter writer = new StreamWriter(batchEntry.Open()))
{
writer.Write(batchFile);
}
//Add the xml file to zip folder
ZipArchiveEntry entry = zip.CreateEntry("nCounterConfig.xml");
using (StreamWriter writer = new StreamWriter(entry.Open()))
{
writer.Write(xdoc.OuterXml);
}
}
zipStream.Position = 0;
return zipStream;
I would like to add a directory with sub directories and files to this memory stream. I found that ZipFile has a method "CreateFromDirectory" which would be ideal except it requires a paramater for an output folder the method also does not have a return type. How can i zip all the files and subfolders in a directory and add them to my memory stream using ZipFile?
something like this
zip.CreateEntry(ZipFile.CreateFromDirectory(
#"C:\morefilestozip\", "",
CompressionLevel.Fastest, true));
I am using this method to compress files and it works great until I get to a file that is 2.4 GB then it gives me an overflow error:
void CompressThis (string inFile, string compressedFileName)
{
FileStream sourceFile = File.OpenRead(inFile);
FileStream destinationFile = File.Create(compressedFileName);
byte[] buffer = new byte[sourceFile.Length];
sourceFile.Read(buffer, 0, buffer.Length);
using (GZipStream output = new GZipStream(destinationFile,
CompressionMode.Compress))
{
output.Write(buffer, 0, buffer.Length);
}
// Close the files.
sourceFile.Close();
destinationFile.Close();
}
What can I do to compress huge files?
You should not to write the whole file to into the memory. Use Stream.CopyTo instead. This method reads the bytes from the current stream and writes them to another stream using a specified buffer size (81920 bytes by default).
Also you don't need to close Stream objects if use using keyword.
void CompressThis (string inFile, string compressedFileName)
{
using (FileStream sourceFile = File.OpenRead(inFile))
using (FileStream destinationFile = File.Create(compressedFileName))
using (GZipStream output = new GZipStream(destinationFile, CompressionMode.Compress))
{
sourceFile.CopyTo(output);
}
}
You can find a more complete example on Microsoft Docs (formerly MSDN).
You're trying to allocate all of this into memory. That just isn't necessary, you can feed the input stream directly into the output stream.
Alternative solution for zip format without allocating memory -
using (var sourceFileStream = new FileStream(this.GetFilePath(sourceFileName), FileMode.Open))
{
using (var destinationStream =
new FileStream(this.GetFilePath(zipFileName), FileMode.Create, FileAccess.ReadWrite))
{
using (var archive = new ZipArchive(destinationStream, ZipArchiveMode.Create, true))
{
var file = archive.CreateEntry(sourceFileName, CompressionLevel.Optimal);
using (var entryStream = file.Open())
{
var fileStream = sourceFileStream;
await fileStream.CopyTo(entryStream);
}
}
}
}
The solution will write directly from input stream to output stream
So I have achieved zipping the files but now I am having this another issues that the zip folder contains empty file. The size of the file zipped is 0 bytes.
This is how I am zipping my file
try
{
var outPutDirectory = AppDomain.CurrentDomain.BaseDirectory;
string logoimage = Path.Combine(outPutDirectory, "images\\error.png");
HttpContext.Current.Response.Clear();
HttpContext.Current.Response.BufferOutput = false;
HttpContext.Current.Response.ContentType = "application/zip";
HttpContext.Current.Response.AddHeader("content-disposition", "attachment; filename=pauls_chapel_audio.zip");
using (MemoryStream ms = new MemoryStream())
{
// create new ZIP archive within prepared MemoryStream
using (ZipArchive zip = new ZipArchive(ms, ZipArchiveMode.Create, true))
{
var demoFile = zip.CreateEntry(logoimage);
// add some files to ZIP archive
}
ms.WriteTo(HttpContext.Current.Response.OutputStream);
}
return true;
}
Another issue is that the zipped folder has the same path as that of the image. So it is like
ZippFolder/A/B/C/image...
I just need
ZipFolder/content
var demoFile = zip.CreateEntry(logoimage);
This creates an entry in the ZIP file that has the name logoimage (i.e. /A/B/C/images/error.png or whatever is the full path).
But you never write to that entry, so it’s empty. Also if you want to have a different path, you should specify it there:
var demoFile = zip.CreateEntry("content\\error.png");
using (StreamWriter writer = new StreamWriter(demoFile.Open()))
using (StreamReader reader = new StreamReader(logoimage))
{
writer.Write(reader.ReadToEnd());
}
Alternatively, you could also skip the StreamWriter completely, and just write to the stream directly:
using (Stream stream = demoFile.Open())
using (StreamReader reader = new StreamReader(logoimage))
{
reader.BaseStream.CopyTo(stream);
}
Btw. you can skip the outer MemoryStream in which you want to write your zip file first and then write that stream to the OutputStream. Instead, you can just write to that stream directly. Just pass it to the ZipFile constructor:
Stream output = HttpContext.Current.Response.OutputStream;
using (ZipArchive zip = new ZipArchive(output, ZipArchiveMode.Create, true))
{
…
}
I am trying to add a file to an existing zip using sharpZibLib in c#.
When run zip gets qverwrite i.e all files within zip gets deleted and only new file is there in a zip.
using (FileStream fileStream = File.Open("D:/Work/Check.zip", FileMode.Open, FileAccess.ReadWrite))
using (ZipOutputStream zipToWrite = new ZipOutputStream(fileStream))
{
zipToWrite.SetLevel(9);
using (FileStream newFileStream = File.OpenRead("D:/Work/file1.txt"))
{
byte[] byteBuffer = new byte[newFileStream.Length - 1];
newFileStream.Read(byteBuffer, 0, byteBuffer.Length);
ZipEntry entry = new ZipEntry("file1.txt");
zipToWrite.PutNextEntry(entry);
zipToWrite.Write(byteBuffer, 0, byteBuffer.Length);
zipToWrite.CloseEntry();
zipToWrite.Finish();
zipToWrite.Close();
}
}
Can anyone tell me whats the issue in above code? Why the zip gets overwitten
Have a look here:
http://wiki.sharpdevelop.net/SharpZipLib_Updating.ashx
you need to call
zipFile.BeginUpdate();
//add file..
zipFile.CommitUpdate();