I'm trying to create a zip stream on the fly with some byte array data and make it download via my MVC action.
But the downloaded file always gives the following corrupted error when opened in windows.
And this error when I try to xtract from 7z
But note that the files extracted from the 7z is not corrupted.
I'm using ZipArchive and the below is my code.
private byte[] GetZippedPods(IEnumerable<POD> pods, long consignmentID)
{
using (var zipStream = new MemoryStream())
{
//Create an archive and store the stream in memory.
using (var zipArchive = new ZipArchive(zipStream, ZipArchiveMode.Create, true))
{
int index = 1;
foreach (var pod in pods)
{
var zipEntry = zipArchive.CreateEntry($"POD{consignmentID}{index++}.png", CompressionLevel.NoCompression);
using (var originalFileStream = new MemoryStream(pod.ByteData))
{
using (var zipEntryStream = zipEntry.Open())
{
originalFileStream.CopyTo(zipEntryStream);
}
}
}
return zipStream.ToArray();
}
}
}
public ActionResult DownloadPOD(long consignmentID)
{
var pods = _consignmentService.GetPODs(consignmentID);
var fileBytes = GetZippedPods(pods, consignmentID);
return File(fileBytes, MediaTypeNames.Application.Octet, $"PODS{consignmentID}.zip");
}
What am I doing wrong here.
Any help would be highly appreciated as I'm struggling with this for a whole day.
Thanks in advance
Move zipStream.ToArray() outside of the zipArchive using.
The reason for your problem is that the stream is buffered. There's a few ways to deal wtih it:
You can set the stream's AutoFlush property to true.
You can manually call .Flush() on the stream.
Or, since it's MemoryStream and you're using .ToArray(), you can simply allow the stream to be Closed/Disposed first (which we've done by moving it outside the using).
I Dispose ZipArchive And error solved
public static byte[] GetZipFile(Dictionary<string, List<FileInformation>> allFileInformations)
{
MemoryStream compressedFileStream = new MemoryStream();
//Create an archive and store the stream in memory.
using (var zipArchive = new ZipArchive(compressedFileStream, ZipArchiveMode.Create, true))
{
foreach (var fInformation in allFileInformations)
{
var files = allFileInformations.Where(x => x.Key == fInformation.Key).SelectMany(x => x.Value).ToList();
for (var i = 0; i < files.Count; i++)
{
ZipArchiveEntry zipEntry = zipArchive.CreateEntry(fInformation.Key + "/" + files[i].FileName);
var caseAttachmentModel = Encoding.UTF8.GetBytes(files[i].Content);
//Get the stream of the attachment
using (var originalFileStream = new MemoryStream(caseAttachmentModel))
using (var zipEntryStream = zipEntry.Open())
{
//Copy the attachment stream to the zip entry stream
originalFileStream.CopyTo(zipEntryStream);
}
}
}
//i added this line
zipArchive.Dispose();
return compressedFileStream.ToArray();
}
}
public void SaveZipFile(){
var zipFileArray = Global.GetZipFile(allFileInformations);
var zipFile = new MemoryStream(zipFileArray);
FileStream fs = new FileStream(path + "\\111.zip",
FileMode.Create,FileAccess.Write);
zipFile.CopyTo(fs);
zipFile.Flush();
fs.Close();
zipFile.Close();
}
I was also having problems with this and I found my issue was not the generation of the archive itself but rather how I was handing my GET request in AngularJS.
This post helped me: how to download a zip file using angular
The key was adding responseType: 'arraybuffer' to my $http call.
factory.serverConfigExportZIP = function () {
return $http({
url: dataServiceBase + 'serverConfigExport',
method: "GET",
responseType: 'arraybuffer'
})
};
you can remove "using" and use Dispose and Close methods
it's work for me
...
zip.Dispose();
zipStream.Close();
return zipStream.ToArray();
I know this is a C# question but for managed C++, delete the ZipArchive^ after you're done with it to fix the error.
ZipArchive^ zar = ZipFile::Open(starget, ZipArchiveMode::Create);
ZipFileExtensions::CreateEntryFromFile(zar, sfile1, "file.txt");
ZipFileExtensions::CreateEntryFromFile(zar, sfile2, "file2.txt");
delete zar;
when i wanted to create zip file directly from MemoryStream which i used for ZipArchive i was getting error ( "unexpected end of data" or zero length file )
there are three points to get ride of this error
set the last parameter of ZipArchive constructor to true ( it leaves to leave stream open after ZipArchive disposed )
call dispose() on ZipArchive and dispose it manually.
create another MemoryStream based on which you set in ZipArchive constructor, by calling ToArray() method.
here is sample code :
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create,))
{
foreach (var s3Object in objectList.S3Objects)
{
var entry = archive.CreateEntry(s3Object.Key, CompressionLevel.NoCompression);
using (var entryStream = entry.Open())
{
var request = new GetObjectRequest { BucketName = command.BucketName, Key = s3Object.Key };
using (var getObjectResponse = await client.GetObjectAsync(request))
{
await getObjectResponse.ResponseStream.CopyToAsync(entryStream);
}
}
}
archive.Dispose();
using (var fileStream = new FileStream(outputFileName, FileMode.Create, FileAccess.Write))
{
var zipFileMemoryStream = new MemoryStream(memoryStream.ToArray());
zipFileMemoryStream.CopyTo(fileStream);
zipFileMemoryStream.Flush();
fileStream.Close();
zipFileMemoryStream.Close();
}
}
}
I had the same problem... In this case I just needed to move the ToArray() (byte[]) from MemoryStream outside the using (var zipArchive = new ZipArchive...
I think it is necessary for using related to ZipArchive to completely close and dispose of the file before converting it into a byte array
Related
In my scenario, I have an use case where I must received a compressed file, do some validations and then find a specific file within the archive that I'll have to handle through a third-party library. I'm having some trouble getting such library to read the file though. This is what I came up with so far:
public async Task ShapeIt(ZipArchive archive)
{
foreach (var entry in archive.Entries)
{
if (Path.GetExtension(entry.FullName).Equals(".shp"))
{
var stream = entry.Open();
using var ms = new MemoryStream();
await stream.CopyToAsync(ms);
ms.Position = 0;
var fileName = Path.GetTempFileName();
try
{
using var fileStream = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.Write,
FileShare.ReadWrite);
var bytes = new byte[ms.Length];
ms.Read(bytes, 0, (int)ms.Length);
fileStream.Write(bytes, 0, bytes.Length);
fileStream.Flush();
fileStream.Close();
var featureSource = new ShapeFileFeatureSource(fileName); // Class from 3rd-party
featureSource.Open();
// Do my stuff with the shapefile
}
finally
{
File.Delete(fileName);
}
}
}
}
Take notice that I'm using the "old way" of copying streams as Stream.CopyTo and Stream.CopyToAsync were creating empty files, explicitly calling fileStream.Close() looks like the only way to get the bytes into the file somehow, but that's beyond my point. Regardless, after closing the stream, upon calling featureSource.Open() my application throws
"The process cannot access the file 'C:\\Users\\me\\AppData\\Local\\Temp\\tmpE926.tmp' because it is
being used by another process."
tmpE926.tmp being different every time, obviously. Also take notice that I'm creating a file because the constructor for ShapeFileFeatureSource demands not a stream, not a byte array, but a path.
A much shorter implementation
public async Task ShapeIt(ZipArchive archive)
{
foreach (var entry in archive.Entries)
{
var tempFile = Path.GetTempFileName();
try
{
entry.ExtractToFile(tempFile, true);
if (Path.GetExtension(entry.FullName).Equals(".shp"))
{
var featureSource = new ShapeFileFeatureSource(tempFile);
featureSource.Open();
var type = featureSource.GetShapeFileType();
}
}
finally
{
File.Delete(tempFile);
}
}
}
will actually amount to the same error. I honestly don't think the problem lies within this library, but rather I'm the one screwing it up somehow. Does anyone have any ideas or should I contact the vendor's (unresponsive) tech support?
Edit: Just in case, this is such library Install-Package ThinkGeo.UI.WebApi but you have to subscribe to evaluate in order to use it.
I could not find package for .NET Core with such classes, so I reproduced it through .NET Framework Nuget package. My answer mostly demonstrates how to deal with streams. It would be hard to tell what is wrong with your code, without having access to library you have
using DotSpatial.Data;
using System.IO;
using System.IO.Compression;
namespace ConsoleApp12
{
class Program
{
static void Main(string[] args)
{
using (var fs = File.OpenRead(#"C:\Users\jjjjjjjjjjjj\Downloads\1270055001_mb_2011_vic_shape.zip"))
using (var zipFile = new ZipArchive(fs))
{
foreach (var entry in zipFile.Entries)
{
if (entry.FullName.EndsWith(".shp"))
{
var tempFile = Path.GetTempFileName();
try
{
using (var entryStream = entry.Open())
using (var newFileStream = File.OpenWrite(tempFile))
{
entryStream.CopyTo(newFileStream);
}
var featureSource = ShapefileFeatureSource.Open(tempFile);
var type = featureSource.ShapeType;
}
finally
{
File.Delete(tempFile);
}
}
}
}
}
}
}
UPD: installed trial version of ThinkGeo library, instead of unauthorized exception it gives me FileNotFoundException with given stacktrace
at ThinkGeo.Core.ValidatorHelper.CheckFileIsExist(String pathFilename)
at ThinkGeo.Core.ShapeFileIndex.xh8=(FileAccess readWriteMode)
^^^^^^^^^^^^^^^^^^^^^^^^^ Are we supposed to have index?
at ThinkGeo.Core.ShapeFile.xh8=(FileAccess readWriteMode)
at ThinkGeo.Core.ShapeFileFeatureSource.WjE=()
at ThinkGeo.Core.ShapeFileFeatureSource.OpenCore()
at ThinkGeo.Core.FeatureSource.Open()
at ConsoleApp20.Program.Main(String[] args) in
C:\Users\jjjjjjjjjjjj\source\repos\ConsoleApp20\ConsoleApp20\Program.cs:line 45
ShapeFileIndex ? So I thought I should dig into this way
var featureSource = new ShapeFileFeatureSource(tempFile);
featureSource.RequireIndex = false; // no luck
featureSource.Open();
I tried to find any reference to the idx file it wants, it has property IndexFilePathName, but unfortunately, I am out of luck. (Also tried different folder, so it is not 'Temp' folder issue)
This code morphed for a couple of days until I reached out to tech support, they tinkered with it a bit and came up with this:
public async Task ProcessFile(IFormFile file)
{
if (!Path.GetExtension(file.FileName).Equals(".zip"))
throw new System.Exception("File should be compressed in '.zip' format");
var filePaths = new List<string>();
using (var stream = new MemoryStream())
{
await file.CopyToAsync(stream);
using (var archive = new ZipArchive(stream, ZipArchiveMode.Read, false))
{
var replaceList = new Dictionary<string, string>();
foreach (ZipArchiveEntry entry in archive.Entries)
{
var tempPath = Path.GetTempFileName();
string key = Path.GetFileNameWithoutExtension(entry.FullName);
string value = Path.GetFileNameWithoutExtension(tempPath);
if (replaceList.ContainsKey(key))
{
value = replaceList[key];
}
else
{
replaceList.Add(key, value);
}
string unzippedPath = Path.Combine(Path.GetDirectoryName(tempPath), value + Path.GetExtension(entry.FullName));
entry.ExtractToFile(unzippedPath, true);
filePaths.Add(unzippedPath);
}
foreach (var unzippedPath in filePaths)
{
if (Path.GetExtension(unzippedPath).Equals(".shp"))
{
// Successfully doing third-party library stuff
}
}
foreach (var unzippedPath in filePaths)
{
if (File.Exists(unzippedPath))
{
File.Delete(unzippedPath);
}
}
}
}
}
It works. I'm happy.
I have files (from 3rd parties) that are being FTP'd to a directory on our server. I download them and process them even 'x' minutes. Works great.
Now, some of the files are .zip files. Which means I can't process them. I need to unzip them first.
FTP has no concept of zip/unzipping - so I'll need to grab the zip file, unzip it, then process it.
Looking at the MSDN zip api, there seems to be no way i can unzip to a memory stream?
So is the only way to do this...
Unzip to a file (what directory? need some -very- temp location ...)
Read the file contents
Delete file.
NOTE: The contents of the file are small - say 4k <-> 1000k.
Zip compression support is built in:
using System.IO;
using System.IO.Compression;
// ^^^ requires a reference to System.IO.Compression.dll
static class Program
{
const string path = ...
static void Main()
{
using(var file = File.OpenRead(path))
using(var zip = new ZipArchive(file, ZipArchiveMode.Read))
{
foreach(var entry in zip.Entries)
{
using(var stream = entry.Open())
{
// do whatever we want with stream
// ...
}
}
}
}
}
Normally you should avoid copying it into another stream - just use it "as is", however, if you absolutely need it in a MemoryStream, you could do:
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0; // rewind
// do something with ms
}
You can use ZipArchiveEntry.Open to get a stream.
This code assumes the zip archive has one text file.
using (FileStream fs = new FileStream(path, FileMode.Open))
using (ZipArchive zip = new ZipArchive(fs) )
{
var entry = zip.Entries.First();
using (StreamReader sr = new StreamReader(entry.Open()))
{
Console.WriteLine(sr.ReadToEnd());
}
}
using (ZipArchive archive = new ZipArchive(webResponse.GetResponseStream()))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Stream s = entry.Open();
var sr = new StreamReader(s);
var myStr = sr.ReadToEnd();
}
}
Looks like here is what you need:
using (var za = ZipFile.OpenRead(path))
{
foreach (var entry in za.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
//your code here
}
}
}
You can use SharpZipLib among a variety of other libraries to achieve this.
You can use the following code example to unzip to a MemoryStream, as shown on their wiki:
using ICSharpCode.SharpZipLib.Zip;
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName) {
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}
Ok so combining all of the above, suppose you want to in a very simple way take a zip file called
"file.zip" and extract it to "C:\temp" folder. (Note: This example was only tested for compress text files) You may need to do some modifications for binary files.
using System.IO;
using System.IO.Compression;
static void Main(string[] args)
{
//Call it like this:
Unzip("file.zip",#"C:\temp");
}
static void Unzip(string sourceZip, string targetPath)
{
using (var z = ZipFile.OpenRead(sourceZip))
{
foreach (var entry in z.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
string uncompressedFile = Path.Combine(targetPath, entry.Name);
File.WriteAllText(uncompressedFile,r.ReadToEnd());
}
}
}
}
I have the following code that generate two kinds of errors. First with the current code I get an exception 'NotSupportedException: This stream from ZipArchiveEntry does not support reading.'. How am I supposed to read the data ?
Furthermore if i use a MemoryStream (as the commented code ) then I can read the data and deserialize correctly but the memorystream i created still remains in memory even if the dispose method has been called on it , causing some memory leaks . Any idea what is wrong with this code ?
void Main()
{
List<Product> products;
using (var s = GetDb().Result)
{
products = Utf8Json.JsonSerializer.Deserialize<List<Product>>(s).ToList();
}
}
// Define other methods and classes here
public static Task<Stream> GetDb()
{
var filepath = Path.Combine("c:/users/tom/Downloads", "productdb.zip");
using (var archive = ZipFile.OpenRead(filepath))
{
var data = archive.Entries.Single(e => e.FullName == "productdb.json");
return Task.FromResult(data.Open());
//using (var reader = new StreamReader(data.Open()))
//{
// var ms = new MemoryStream();
// data.Open().CopyTo(ms);
// ms.Seek(0, SeekOrigin.Begin);
// return Task.FromResult((Stream)ms);
//}
}
}
With the commented code you open the stream into a reader, don't use the reader, then open the stream again and copy over to the memory stream without closing the second opened stream.
It is the second opened stream that remains in memory, not the MemoryStream.
Refactor
public static async Task<Stream> GetDb() {
var filepath = Path.Combine("c:/users/tom/Downloads", "productdb.zip");
using (var archive = ZipFile.OpenRead(filepath)) {
var entry = archive.Entries.Single(e => e.FullName == "productdb.json");
using (var stream = entry.Open()) {
var ms = new MemoryStream();
await stream.CopyToAsync(ms);
return ms;
}
}
}
I'm trying to serialize an instance of a class (let's call it Car) into an xml and compress it in memory to a zip file with a single file entry in it.
I'm using the System.IO.Compression.ZipArchive class to do it:
private byte[] CompressCar(Car car)
{
using (var carStream = new MemoryStream())
using (var zipStream = new MemoryStream())
using (var archive = new ZipArchive(zipStream, ZipArchiveMode.Create))
{
new XmlSerializer(typeof(Car)).Serialize(carStream, car);
ZipArchiveEntry entry = archive.CreateEntry("car.xml", CompressionLevel.Optimal);
using (var zippedFileStream = entry.Open())
{
carStream.Seek(0, SeekOrigin.Begin);
carStream.CopyTo(zippedFileStream);
}
return zipStream.ToArray();
}
}
When I save the compressed bytes to a file and later try to open it with Windows Explorer I get an error:
What am I doing wrong here?
I looked for other StackOverflow entries but I just couldn't find anything that would solve my problem. I want to compress it in memory rather than using a temporary file.
You need to dispose the ZipArchive before returning the underlying zipStream.ToArray(). E.g., you could extract the following helper method:
public static class SerializationExtensions
{
public static byte[] ToCompressedXmlZipArchive<T>(T root, string entryName)
{
using (var zipStream = new MemoryStream())
{
using (var archive = new ZipArchive(zipStream, ZipArchiveMode.Create))
{
var entry = archive.CreateEntry(entryName, CompressionLevel.Optimal);
using (var zippedFileStream = entry.Open())
{
new XmlSerializer(typeof(T)).Serialize(zippedFileStream, root); // Possibly use root.GetType() instead of typeof(T)
}
}
return zipStream.ToArray();
}
}
}
And then your method becomes:
private byte[] CompressCar(Car car)
{
return SerializationExtensions.ToCompressedXmlZipArchive(car, "car.xml");
}
I have files (from 3rd parties) that are being FTP'd to a directory on our server. I download them and process them even 'x' minutes. Works great.
Now, some of the files are .zip files. Which means I can't process them. I need to unzip them first.
FTP has no concept of zip/unzipping - so I'll need to grab the zip file, unzip it, then process it.
Looking at the MSDN zip api, there seems to be no way i can unzip to a memory stream?
So is the only way to do this...
Unzip to a file (what directory? need some -very- temp location ...)
Read the file contents
Delete file.
NOTE: The contents of the file are small - say 4k <-> 1000k.
Zip compression support is built in:
using System.IO;
using System.IO.Compression;
// ^^^ requires a reference to System.IO.Compression.dll
static class Program
{
const string path = ...
static void Main()
{
using(var file = File.OpenRead(path))
using(var zip = new ZipArchive(file, ZipArchiveMode.Read))
{
foreach(var entry in zip.Entries)
{
using(var stream = entry.Open())
{
// do whatever we want with stream
// ...
}
}
}
}
}
Normally you should avoid copying it into another stream - just use it "as is", however, if you absolutely need it in a MemoryStream, you could do:
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0; // rewind
// do something with ms
}
You can use ZipArchiveEntry.Open to get a stream.
This code assumes the zip archive has one text file.
using (FileStream fs = new FileStream(path, FileMode.Open))
using (ZipArchive zip = new ZipArchive(fs) )
{
var entry = zip.Entries.First();
using (StreamReader sr = new StreamReader(entry.Open()))
{
Console.WriteLine(sr.ReadToEnd());
}
}
using (ZipArchive archive = new ZipArchive(webResponse.GetResponseStream()))
{
foreach (ZipArchiveEntry entry in archive.Entries)
{
Stream s = entry.Open();
var sr = new StreamReader(s);
var myStr = sr.ReadToEnd();
}
}
Looks like here is what you need:
using (var za = ZipFile.OpenRead(path))
{
foreach (var entry in za.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
//your code here
}
}
}
You can use SharpZipLib among a variety of other libraries to achieve this.
You can use the following code example to unzip to a MemoryStream, as shown on their wiki:
using ICSharpCode.SharpZipLib.Zip;
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName) {
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}
Ok so combining all of the above, suppose you want to in a very simple way take a zip file called
"file.zip" and extract it to "C:\temp" folder. (Note: This example was only tested for compress text files) You may need to do some modifications for binary files.
using System.IO;
using System.IO.Compression;
static void Main(string[] args)
{
//Call it like this:
Unzip("file.zip",#"C:\temp");
}
static void Unzip(string sourceZip, string targetPath)
{
using (var z = ZipFile.OpenRead(sourceZip))
{
foreach (var entry in z.Entries)
{
using (var r = new StreamReader(entry.Open()))
{
string uncompressedFile = Path.Combine(targetPath, entry.Name);
File.WriteAllText(uncompressedFile,r.ReadToEnd());
}
}
}
}