I have a zip file at a URI (such as http://www.abc.com/a.zip) that I would like to open and save a file from it to disk. Is there a way in C# to open it without saving it to disk and then saving a file from it to disk?
Thanks,
Sachin
Use ZipFile .Net Framework 4.5 class or DotNetZip API.
ZipFile.ExtractToDirectory(zipPath, extractPath);
EDIT: You can prepare a stream or obtain byte array of URL via method of WebClient class.
string urlStr = "https://xyz.com/sample.zip";
using (WebClient client = new WebClient())
{
byte []bytes=client.DownloadData(urlStr);
using (MemoryStream ms = new MemoryStream(bytes))
{
using (ZipFile zip = ZipFile.Read(ms))
{
zip.ExtractAll(#"C:\csnet");
}
}
}
The example of the extracting the zip file without saving the archive to the file system using DotNetZip.
private static void ExtractFromUrl(Uri uri, string directoryPath)
{
using (var webClient = new WebClient())
{
var data = webClient.DownloadData(uri);
using (var memoryStream = new MemoryStream(data))
using (var zipFile = ZipFile.Read(memoryStream))
{
zipFile.ExtractAll(directoryPath);
}
}
}
Related
Whenever I try to upload a file to the SFTP server with the .csv file extension the only thing within that file is System.IO.MemoryStream. If it's a .txt extension it will have all the values in the file. I can manually convert the .txt to .csv and it will be fine. Is it possible to upload it directly to the SFTP server as a CSV file?
The SFTP Service is using the SSH.NET library by Renci.
Using statement:
using (var stream = csvFileWriter.Write(data, new CsvMapper()))
{
byte[] file = Encoding.UTF8.GetBytes(stream.ToString());
sftpService.Put(SftpCredential.Credentials.Id, file, $"/file.csv");
}
SFTP service:
public void Put(int credentialId, byte[] source, string destination)
{
using (SftpClient client = new SftpClient(GetConnectionInfo(credentialId)))
{
ConnectClient(client);
using (MemoryStream memoryStream = new MemoryStream(source))
{
client.BufferSize = 4 * 1024; // bypass Payload error large files
client.UploadFile(memoryStream, destination);
}
DisconnectClient(client);
}
Solution:
The csvFilerWriter I was using returned a Stream not a MemoryStream, so by switching the csvFileWriter and CsvPut() over to MemoryStream it worked.
Updated using statement:
using (var stream = csvFileWriter.Write(data, new CsvMapper()))
{
stream.Position = 0;
sftpService.CsvPut(SftpCredential.credemtoa;s.Id, stream, $"/file.csv");
}
Updated SFTP service:
public void CsvPut(int credentialId, MemoryStream source, string destination)
{
using (SftpClient client = new SftpClient(GetConnectionInfo(credentialId)))
{
ConnectClient(client);
client.BufferSize = 4 * 1024; //bypass Payload error large files
client.UploadFile(source, destination);
DisconnectClient(client);
}
}
It looks like the csvFileWriter.Write already returns MemoryStream. And its ToString returns "System.IO.MemoryStream" string. That's the root source of your problem.
Aditionally, as you already have the MemoryStream, its an overkill to copy it to yet another MemoryStream, upload it directly. You are copying the data over and over again, it's just a waste of memory.
Like this:
var stream = csvFileWriter.Write(data, new CsvMapper());
stream.Position = 0;
client.UploadFile(stream, destination);
See also:
Upload data from memory to SFTP server using SSH.NET
When uploading memory stream with contents created by csvhelper using SSH.NET to SFTP server, the uploaded file is empty
A simple test code to upload in-memory data:
var stream = new MemoryStream();
stream.Write(Encoding.UTF8.GetBytes("this is test"));
stream.Position = 0;
using (var client = new SftpClient("example.com", "username", "password"))
{
client.Connect();
client.UploadFile(stream, "/remote/path/file.txt");
}
You can avoid the unnecessary using of memory stream like this:
using (var sftp = new SftpClient(GetConnectionInfo(SftpCredential.GetById(credentialId).Id))))
{
sftp.Connect();
using (var uplfileStream = System.IO.File.OpenRead(fileName))
{
sftp.UploadFile(uplfileStream, fileName, true);
}
sftp.Disconnect();
}
I have a piece of code that allows to decompress a byte array:
public static byte[] Decompress(this byte[] data)
{
using (ZipFile zout = ZipFile.Read(data))
{
ZipEntry entry = zout.FirstOrDefault();
Assert.ObjectIsNotNull(entry, "Unable to find default ZIP entry");
MemoryStream zos = new MemoryStream();
entry.Extract(zos);
return zos.ToArray();
}
}
I upgraded to the latest version of Ionic.zip and now I am getting the following error:
Cannot convert byte[] to string.
The overload ZipFile.Read(byte[]) is no longer available in the most recent version.
How can I read a zip file from a byte array?
The ZipFile.Read method takes either a filename or a stream to read, so you need to provide a stream for it to read:
using (MemoryStream stream = new MemoryStream(data))
using (ZipFile zout = ZipFile.Read(stream))
{
// ....
You can use the built-in ZipArchive class in System.IO.Commpression.
using(var stream = new MemoryStream(data))
{
using(var archive = new ZipArchive(stream))
{
// Use the archive
}
}
ZipArchive
https://msdn.microsoft.com/en-us/library/hh158268(v=vs.110).aspx
MemoryStream
https://msdn.microsoft.com/en-us/library/e55f3s5k(v=vs.110).aspx
You will need to add a reference to System.IO.Compression, it is not in mscorlib.
I am writing a REST API which will take in a JSON request object. The request object will have to be serialized to a file in JSON format; the file has to be compressed into a zip file and the ZIP file has to be posted to another service, for which I would have to deserialize the ZIP file. All this because the service I have to call expects me to post data as ZIP file. I am trying to see if I can avoid disk IO. Is there a way to directly convert the object into a byte array representing ZIP content in-memory instead of all the above steps?
Note : I'd prefer accomplishing this using .net framework libraries (as against external libraries)
Yes, it is possible to create a zip file completely on memory, here is an example using SharpZip Library (Update: A sample using ZipArchive added at the end):
public static void Main()
{
var fileContent = Encoding.UTF8.GetBytes(
#"{
""fruit"":""apple"",
""taste"":""yummy""
}"
);
var zipStream = new MemoryStream();
var zip = new ZipOutputStream(zipStream);
AddEntry("file0.json", fileContent, zip); //first file
AddEntry("file1.json", fileContent, zip); //second file (with same content)
zip.Close();
//only for testing to see if the zip file is valid!
File.WriteAllBytes("test.zip", zipStream.ToArray());
}
private static void AddEntry(string fileName, byte[] fileContent, ZipOutputStream zip)
{
var zipEntry = new ZipEntry(fileName) {DateTime = DateTime.Now, Size = fileContent.Length};
zip.PutNextEntry(zipEntry);
zip.Write(fileContent, 0, fileContent.Length);
zip.CloseEntry();
}
You can obtain SharpZip using Nuget command PM> Install-Package SharpZipLib
Update:
Note : I'd prefer accomplishing this using .net framework libraries (as against external libraries)
Here is an example using Built-in ZipArchive from System.IO.Compression.Dll
public static void Main()
{
var fileContent = Encoding.UTF8.GetBytes(
#"{
""fruit"":""apple"",
""taste"":""yummy""
}"
);
var zipContent = new MemoryStream();
var archive = new ZipArchive(zipContent, ZipArchiveMode.Create);
AddEntry("file1.json",fileContent,archive);
AddEntry("file2.json",fileContent,archive); //second file (same content)
archive.Dispose();
File.WriteAllBytes("testa.zip",zipContent.ToArray());
}
private static void AddEntry(string fileName, byte[] fileContent,ZipArchive archive)
{
var entry = archive.CreateEntry(fileName);
using (var stream = entry.Open())
stream.Write(fileContent, 0, fileContent.Length);
}
You could use the GZipStream class along with MemoryStream.
A quick example:
using System.IO;
using System.IO.Compression;
//Put JSON into a MemoryStream
var theJson = "Your JSON Here";
var jsonStream = new MemoryStream();
var jsonStreamWriter = new StreamWriter(jsonStream);
jsonStreamWriter.Write(theJson);
jsonStreamWriter.Flush();
//Reset stream so it points to the beginning of the JSON
jsonStream.Seek(0, System.IO.SeekOrigin.Begin);
//Create stream to hold your zipped JSON
var zippedStream = new MemoryStream();
//Zip JSON and put it in zippedStream via compressionStream.
var compressionStream = new GZipStream(zippedStream, CompressionLevel.Optimal);
jsonStream.CopyTo(compressionStream);
//Reset zipped stream to point at the beginning of data
zippedStream.Seek(0, SeekOrigin.Begin);
//Get ByteArray with zipped JSON
var zippedJsonBytes = zippedStream.ToArray();
You should try the ZipArchive Class streaming to a MemoryStream Class
Yes. You can return it as a binary stream. Depending on the language, you can use special libraries. You will also need libraries on the client.
I am trying to upload a compressed GZipStream to sftp server using ssh.net library. The problem is that when I create the GZipStream, it can not be read any more. Below is my code:
using (SftpClient client = new SftpClient(connectionInfo))
{
client.Connect();
client.ChangeDirectory("/upload");
var uploadFileDirectory = client.WorkingDirectory + "\testXml.xml.gz";
using (GZipStream gzs = new GZipStream(stream, CompressionLevel.Fastest))
{
stream.CopyTo(gzs);
client.UploadFile(gzs, "text.xml.gz");
}
}
The SftpClient's UploadFile takes a stream and I need to upload the GzipStream that is being compressed (without storing to a local drive and then read it again). But the GZipStream doesn't allow read when it is compressed. I tried doign the upload outside the gzipstream using clause and it says that the stream can not be accessed.
How can I approach this? Is it even possible to do it directly this way or do I need to write it to local drive and then upload it...
For future reference, I manage to find how to do this. You can't read the gZipStream on Compression mode but you can get create another MemoryStream of previous stream's bytes like this:
using (SftpClient client = new SftpClient(connectionInfo))
{
client.Connect();
client.ChangeDirectory("/upload");
using (MemoryStream outputStream = new MemoryStream())
{
using (var gzip = new GZipStream(outputStream, CompressionLevel.Fastest))
{
stream.CopyTo(gzip);
}
using (Stream stm = new MemoryStream(outputStream.ToArray()))
{
client.UploadFile(stm,"txt.gz");
}
}
}
Is there a way to create a Zip archive that contains multiple files, when the files are currently in memory? The files I want to save are really just text only and are stored in a string class in my application. But I would like to save multiple files in a single self-contained archive. They can all be in the root of the archive.
It would be nice to be able to do this using SharpZipLib.
Use ZipEntry and PutNextEntry() for this. The following shows how to do it for a file, but for an in-memory object just use a MemoryStream
FileStream fZip = File.Create(compressedOutputFile);
ZipOutputStream zipOStream = new ZipOutputStream(fZip);
foreach (FileInfo fi in allfiles)
{
ZipEntry entry = new ZipEntry((fi.Name));
zipOStream.PutNextEntry(entry);
FileStream fs = File.OpenRead(fi.FullName);
try
{
byte[] transferBuffer[1024];
do
{
bytesRead = fs.Read(transferBuffer, 0, transferBuffer.Length);
zipOStream.Write(transferBuffer, 0, bytesRead);
}
while (bytesRead > 0);
}
finally
{
fs.Close();
}
}
zipOStream.Finish();
zipOStream.Close();
Using SharpZipLib for this seems pretty complicated. This is so much easier in DotNetZip. In v1.9, the code looks like this:
using (ZipFile zip = new ZipFile())
{
zip.AddEntry("Readme.txt", stringContent1);
zip.AddEntry("readings/Data.csv", stringContent2);
zip.AddEntry("readings/Index.xml", stringContent3);
zip.Save("Archive1.zip");
}
The code above assumes stringContent{1,2,3} contains the data to be stored in the files (or entries) in the zip archive. The first entry is "Readme.txt" and it is stored in the top level "Directory" in the zip archive. The next two entries are stored in the "readings" directory in the zip archive.
The strings are encoded in the default encoding. There is an overload of AddEntry(), not shown here, that allows you to explicitly specify the encoding to use.
If you have the content in a stream or byte array, not a string, there are overloads for AddEntry() that accept those types. There are also overloads that accept a Write delegate, a method of yours that is invoked to write data into the zip. This works for easily saving a DataSet into a zip file, for example.
DotNetZip is free and open source.
This function should create a byte array from a stream of data: I've created a simple interface for handling files for simplicity
public interface IHasDocumentProperties
{
byte[] Content { get; set; }
string Name { get; set; }
}
public void CreateZipFileContent(string filePath, IEnumerable<IHasDocumentProperties> fileInfos)
{
using (var memoryStream = new MemoryStream())
{
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
foreach(var fileInfo in fileInfos)
{
var entry = zipArchive.CreateEntry(fileInfo.Name);
using (var entryStream = entry.Open())
{
entryStream.Write(fileInfo.Content, 0, fileInfo.Content.Length);
}
}
}
using (var fileStream = new FileStream(filePath, FileMode.OpenOrCreate, System.IO.FileAccess.Write))
{
memoryStream.Position = 0;
memoryStream.CopyTo(fileStream);
}
}
}
Yes, you can use SharpZipLib to do this - when you need to supply a stream to write to, use a MemoryStream.
I come across this problem, using the MSDN example I created this class:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO.Packaging;
using System.IO;
public class ZipSticle
{
Package package;
public ZipSticle(Stream s)
{
package = ZipPackage.Open(s, FileMode.Create);
}
public void Add(Stream stream, string Name)
{
Uri partUriDocument = PackUriHelper.CreatePartUri(new Uri(Name, UriKind.Relative));
PackagePart packagePartDocument = package.CreatePart(partUriDocument, "");
CopyStream(stream, packagePartDocument.GetStream());
stream.Close();
}
private static void CopyStream(Stream source, Stream target)
{
const int bufSize = 0x1000;
byte[] buf = new byte[bufSize];
int bytesRead = 0;
while ((bytesRead = source.Read(buf, 0, bufSize)) > 0)
target.Write(buf, 0, bytesRead);
}
public void Close()
{
package.Close();
}
}
You can then use it like this:
FileStream str = File.Open("MyAwesomeZip.zip", FileMode.Create);
ZipSticle zip = new ZipSticle(str);
zip.Add(File.OpenRead("C:/Users/C0BRA/SimpleFile.txt"), "Some directory/SimpleFile.txt");
zip.Add(File.OpenRead("C:/Users/C0BRA/Hurp.derp"), "hurp.Derp");
zip.Close();
str.Close();
You can pass a MemoryStream (or any Stream) to ZipSticle.Add such as:
FileStream str = File.Open("MyAwesomeZip.zip", FileMode.Create);
ZipSticle zip = new ZipSticle(str);
byte[] fileinmem = new byte[1000];
// Do stuff to FileInMemory
MemoryStream memstr = new MemoryStream(fileinmem);
zip.Add(memstr, "Some directory/SimpleFile.txt");
memstr.Close();
zip.Close();
str.Close();
Note this answer is outdated; since .Net 4.5, the ZipArchive class allows zipping files in-memory. See johnny 5's answer below for how to use it.
You could also do it a bit differently, using a Serializable object to store all strings
[Serializable]
public class MyStrings {
public string Foo { get; set; }
public string Bar { get; set; }
}
Then, you could serialize it into a stream to save it.
To save on space you could use GZipStream (From System.IO.Compression) to compress it. (note: GZip is stream compression, not an archive of multiple files).
That is, of course if what you need is actually to save data, and not zip a few files in a specific format for other software.
Also, this would allow you to save many more types of data except strings.
I was utilizing Cheeso's answer by adding MemoryStreams as the source of the different Excel files. When I downloaded the zip, the files had nothing in them. This could be the way we were getting around trying to create and download a file over AJAX.
To get the contents of the different Excel files to be included in the Zip, I had to add each of the files as a byte[].
using (var memoryStream = new MemoryStream())
using (var zip = new ZipFile())
{
zip.AddEntry("Excel File 1.xlsx", excelFileStream1.ToArray());
zip.AddEntry("Excel File 2.xlsx", excelFileStream2.ToArray());
// Keep the file off of disk, and in memory.
zip.Save(memoryStream);
}
Use a StringReader to read from your string objects and expose them as Stream s.
That should make it easy to feed them to your zip-building code.