The following code works well with small files, like 100MB, but it throws a System.OutOfMemoryException for bigger files, like 400MB. I'm using the NotNetZip as dll to get file as zip.
This is my code:
string pathGetDoc = pathDocs + "\\" + informe.NickName + "\\" + getMesActual() + "\\" + informe.Name;
string fileName = informe.Name;
System.Net.WebClient wc = new System.Net.WebClient();
wc.OpenRead(pathGetDoc);
int bytes_total = Convert.ToInt32(wc.ResponseHeaders["Content-Length"].ToString());
if(bytes_total >= 100000000)
{
using (ZipFile zip = new ZipFile())
{
zip.AddFile(pathGetDoc, fileName);
zip.CompressionMethod = CompressionMethod.BZip2;
zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestCompression;
using (MemoryStream memoryStream = new MemoryStream())
{
zip.Save(memoryStream);
return File(memoryStream.ToArray(), "application/zip", "z.zip");
}
}
}
As you can see, I have a IF to check the size of the file, this work good but when the process goes to save .zip file, I have the error System.OutOfMemoryException
Related
I have issue when action open file follow format .gz/.tar.gz
This is image error
This is code:
public void Check(string path)
{
foreach (var fileName in Directory.GetFiles(path))
{
if (Path.GetExtension(fileName) == ".gz" && Path.GetFileNameWithoutExtension(fileName).Contains("tar"))
{
Console.WriteLine(fileName);
ExtractTGZ(fileName, Path.GetDirectoryName(fileName) + #"\" + Path.GetFileNameWithoutExtension(fileName));
System.IO.File.Delete(fileName);
}
}
}
The System.IO.Compression namespace contains the following types for compressing and decompressing files and streams. You can also use these types to read and modify the contents of a compressed file.
string fileName = Guid.NewGuid() + ifile.FileName;
string path = Path.Combine(FilesTempPath, fileName);
using (var fileStream = new FileStream(path, FileMode.Create))
{
ifile.CopyTo(fileStream);
}
var FileZipPath = FilesTempPath + "\\" + fileName;
ZipFile.ExtractToDirectory(FileZipPath, FilesTempPath);
The following example from MSDN shows how to use the GZipStream class to compress and decompress a directory of files.
Requirement:
1) creating split zips file in multiple segments(say size - 1 GB/500 MB), so that they can be downloaded through browser. The total zip volume of all the segments could exceed 10 GB
2) the zip content could be multiple files or a folder containing sub folders and files
3) the content of the file are read from Cloud in the form of stream. The meta information for the files(like folder hierarchy) are locally available
I am using DotNetZip library to achieve the task. The code is as following:
long length = default(long);
Stream fileReadStream;
long Space = default(long);
string tempZipFile = string.Empty;
FileZipStatus oldStatue = new FileZipStatus();
byte[] Buffer = new byte[1024 * 1024];
if (zipFileName != null && !zipFileName.ToUpper().EndsWith(".ZIP")) zipFileName += ".zip";
string strTempFolder = "";
using (Ionic.Zip.ZipFile zip = new Ionic.Zip.ZipFile())
{
try
{
strTempFolderPath = tempZipOutPutFilePath + "\\";
string strTempFolderName = DateTime.Now.Ticks.ToString();
strTempFolder = strTempFolderPath + strTempFolderName;
if (userFileList.Count > 0)
{
if (Directory.Exists(strTempFolder))
{
Directory.Delete(strTempFolder);
}
Directory.CreateDirectory(strTempFolder);
}
foreach (UserFile userFile in userFileList)
{
WebResponse response = null;
try
{
WebRequest request = null;
IDictionary<string, object> _dictionary = new Dictionary<string, object>();
/// First
FileSystemEnum fileSysEnum = FileSystemBase.GetFileSystemEnumByStorageId(userFile.StorageId);
IFileSystemLib ifileSystemLocal = FileSystemFactory.GetSpecificInstance(fileSysEnum);
fileReadStream = ifileSystemLocal.GetFile(userFile.FilePath, userFile.GuidName, ref request, ref response, _dictionary);
long filesize = default(long);
long.TryParse(ifileSystemLocal.GetFileContentLength(userFile.FilePath, userFile.GuidName).ToString(), out filesize);
Space = (Space > default(long)) ? (Space + filesize) : filesize;
//Now we have to store the data, so that we must access the file
int dataToRead;
FileStream writeStream = new FileStream(strTempFolder + "\\" + userFile.FileName, FileMode.Create, FileAccess.Write);
while ((dataToRead = fileReadStream.Read(Buffer, 0, Buffer.Length)) > 0)
{
writeStream.Write(Buffer, 0, dataToRead);
}
writeStream.Close();
zip.AddFile(strTempFolder + "\\" + userFile.FileName, userFile.RelativePath);
fileReadStream.Close();
}
catch (Exception ex)
{
LogManager.Trace(ex, "ZIpping Block - ZIPFileName", zipFileName + "File to zip" + userFile.GuidName);
}
finally
{
if (response != null) response.Close();
}
}
}
catch (Exception ex)
{
_currentStatus = FileZipStatus.NotAvailable;
oldStatue = UpdateZipStatus(ObjectZipID, Space, FileZipStatus.Failed);
throw ex;
}
finally
{
}
try
{
zip.Comment = "This zip was created at " + System.DateTime.Now.ToString("G");
zip.MaxOutputSegmentSize = 200 * 1024 * 1024; // 200 mb
zip.Save(strTempFolderPath + "\\" + zipFileName);
oldStatue = UpdateZipStatus(ObjectZipID, Space, FileZipStatus.Available);
length = new FileInfo(strTempFolderPath + "\\" + zipFileName).Length;
_currentStatus = FileZipStatus.Available;
// deleting temp folder
Directory.Delete(strTempFolder, true);
}
catch (Exception ex)
{
_currentStatus = FileZipStatus.NotAvailable;
oldStatue = UpdateZipStatus(ObjectZipID, Space, FileZipStatus.Failed);
length = default(long);
throw ex;
}
}
There are a limitation of the DotNetZip libray used in the above code.
It either needs
a) files saved on disk as input. In that case folder hierarchy information could be passed for each file.
or
2) if stream is passed as input, folder hierarchy information could NOT be passed for file.
I need to pass in the folder hierarchy information for each file as well as read the input from stream. As the zip content could be huge(could exceed 10 GB),
do not want to save the files on temporary storage in web server. Can Anyone help like how to pass folder hierarchy when creating zip file? thanks
i got the solution. here is the code
private void button2_Click(object sender, EventArgs e)
{
using (SqlConnection sqlConn = new SqlConnection(#"Data Source=BBATRIDIP\SQLSERVER2008R2;Initial Catalog=test;Integrated Security=True"))
{
string query = String.Format(#"SELECT [FilePath],[FileName],[FileData] FROM [TestTable]");
SqlCommand cmd = new SqlCommand(query, sqlConn);
cmd.Connection.Open();
System.IO.MemoryStream memStream = null;
ZipFile zip = new ZipFile();
zip.MaxOutputSegmentSize = 1024 * 1024; // 1MB each segment size would be
// the above line would split zip file into multiple files and each file
//size would be 1MB
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
byte[] data = (byte[])reader["FileData"];
memStream = new System.IO.MemoryStream(data);
string strFile = reader["FilePath"].ToString() + "\\" + reader["FileName"].ToString();
ZipEntry ze = zip.AddEntry(strFile, memStream);
}
}
zip.Save(#"e:\MyCustomZip.zip");
memStream.Dispose();
MessageBox.Show("Job Done");
// here u can save the zip in memory stream also there is a overload insteaa of saving in HD
}
}
this approach stores the zip content in memory. Hence, when the zip content is huge, say exceeds 5 GB then then it crashes. Need to write to fileOutputStream mapped to physical file
I am trying to download a file usng FTP within a C# console application, but even though I now the paths are correct I always get an error saying "550 file not found".
Is there any way, to return the current path (once connected to the server)?
// lade datei von FTP server
string ftpfullpath = "ftp://" + Properties.Settings.Default.FTP_Server + Properties.Settings.Default.FTP_Pfad + "/" + Properties.Settings.Default.FTP_Dateiname;
Console.WriteLine("Starte Download von: " + ftpfullpath);
using (WebClient request = new WebClient())
{
request.Credentials = new NetworkCredential(Properties.Settings.Default.FTP_User, Properties.Settings.Default.FTP_Passwort);
byte[] fileData = request.DownloadData(ftpfullpath);
using (FileStream file = File.Create(#path + "/tmp/" + Properties.Settings.Default.FTP_Dateiname))
{
file.Write(fileData, 0, fileData.Length);
file.Close();
}
Console.WriteLine("Download abgeschlossen!");
}
EDIT
My mistake. Fixed the filepath, still getting the same error. But if I connect with FileZilla that's the exact file path.
Finally found a solution by using System.Net.FtpClient (https://netftp.codeplex.com/releases/view/95632) and using the following code.
// aktueller pfad
string apppath = Directory.GetCurrentDirectory();
Console.WriteLine("Bereite Download von FTP Server vor!");
using (var ftpClient = new FtpClient())
{
ftpClient.Host = Properties.Settings.Default.FTP_Server;
ftpClient.Credentials = new NetworkCredential(Properties.Settings.Default.FTP_User, Properties.Settings.Default.FTP_Passwort);
var destinationDirectory = apppath + "\\Input";
ftpClient.Connect();
var destinationPath = string.Format(#"{0}\{1}", destinationDirectory, Properties.Settings.Default.FTP_Dateiname);
Console.WriteLine("Starte Download von " + Properties.Settings.Default.FTP_Dateiname + " nach " + destinationPath);
using (var ftpStream = ftpClient.OpenRead(Properties.Settings.Default.FTP_Pfad + "/" + Properties.Settings.Default.FTP_Dateiname))
using (var fileStream = File.Create(destinationPath , (int)ftpStream.Length))
{
var buffer = new byte[8 * 1024];
int count;
while ((count = ftpStream.Read(buffer, 0, buffer.Length)) > 0)
{
fileStream.Write(buffer, 0, count);
}
}
}
I think your filename is wrong. Your first line writes a different name than what you set to ftpfullpath. You us FTP_Dateiname on the first line but FTP_Pfad when you set ftpfullpath.
To see what's actually happening move your first line after 'string ftpfullpath...')
and change it to Console.WriteLine("Starte Download von: " + ftpfullpath);
I am developing wpf application. I am using sharpziplib to compress and decompress files. I am easily decompress the .zip files using following code
public static void UnZip(string SrcFile, string DstFile, string safeFileName, int bufferSize)
{
//ICSharpCode.SharpZipLib.Zip.UseZip64.Off;
FileStream fileStreamIn = new FileStream(SrcFile, FileMode.Open, FileAccess.Read);
ZipInputStream zipInStream = new ZipInputStream(fileStreamIn);
string rootDirectory = string.Empty;
if (safeFileName.Contains(".zip"))
{
rootDirectory = safeFileName.Replace(".zip", string.Empty);
}
else
{
rootDirectory = safeFileName;
}
Directory.CreateDirectory(App.ApplicationPath + rootDirectory);
while (true)
{
ZipEntry entry = zipInStream.GetNextEntry();
if (entry == null)
break;
if (entry.Name.Contains("/"))
{
string[] folders = entry.Name.Split('/');
string lastElement = folders[folders.Length - 1];
var folderList = new List<string>(folders);
folderList.RemoveAt(folders.Length - 1);
folders = folderList.ToArray();
string folderPath = "";
foreach (string str in folders)
{
folderPath = folderPath + "/" + str;
if (!Directory.Exists(App.ApplicationPath + rootDirectory + "/" + folderPath))
{
Directory.CreateDirectory(App.ApplicationPath + rootDirectory + "/" + folderPath);
}
}
if (!string.IsNullOrEmpty(lastElement))
{
folderPath = folderPath + "/" + lastElement;
WriteToFile(DstFile + rootDirectory + #"\" + folderPath, bufferSize, zipInStream, rootDirectory, entry);
}
}
else
{
WriteToFile(DstFile + rootDirectory + #"\" + entry.Name, bufferSize, zipInStream, rootDirectory, entry);
}
}
zipInStream.Close();
fileStreamIn.Close();
}
private static void WriteToFile(string DstFile, int bufferSize, ZipInputStream zipInStream, string rootDirectory, ZipEntry entry)
{
FileStream fileStreamOut = new FileStream(DstFile, FileMode.OpenOrCreate, FileAccess.Write);
int size;
byte[] buffer = new byte[bufferSize];
do
{
size = zipInStream.Read(buffer, 0, buffer.Length);
fileStreamOut.Write(buffer, 0, size);
} while (size > 0);
fileStreamOut.Close();
}
But the same code is not working with .bz2 files. It is giving error at line
ZipEntry entry = zipInStream.GetNextEntry();
The error is - Wrong Local header signature: 0x26594131. How should I decompress the .bz2 file ? Can you please provide me any code or link through which I can resolve the above issue ?
While you use a ZipInputStream for .zip files, you should use a BZip2InputStream for .bz2 files (and GZipInputStream for .gz files etc.).
Unlike Zip (and RAR and tar), bz2 and gzip are just byte stream compressors. They have no concept of a container format like the aforementioned, and hence why it fails on GetNextEntry. (In other words, bz2 and gzip will only have 1 entry at most).
I need help in converting a file received from a jquery ajax to byte array. I'm using a plugin called ajaxfileupload then from a jquery ajax call I send a file from a fileupload control to a handler. Here is my
handler code:
if (context.Request.Files.Count > 0)
{
string path = context.Server.MapPath("~/Temp");
if (!Directory.Exists(path))
Directory.CreateDirectory(path);
var file = context.Request.Files[0];
string fileName;
if (HttpContext.Current.Request.Browser.Browser.ToUpper() == "IE")
{
string[] files = file.FileName.Split(new char[] { '\\' });
fileName = files[files.Length - 1];
}
else
{
fileName = file.FileName;
}
string fileType = file.ContentType;
string strFileName = fileName;
FileStream fs = new FileStream("~/Temp/" + strFileName, FileMode.Open, FileAccess.Read);
BinaryReader br = new BinaryReader(fs);
Byte[] imagebytes = br.ReadBytes((Int32)fs.Length);
br.Close();
fs.Close();
DBAccess dbacc = new DBAccess();
dbacc.saveImage(imagebytes);
string msg = "{";
msg += string.Format("error:'{0}',\n", string.Empty);
msg += string.Format("msg:'{0}'\n", strFileName);
msg += "}";
context.Response.Write(msg);
}
I'm saving the file to a folder within a project then trying to retrieve that file and save it to the database. I can assure you that the image is being saved to the temp folder. The problem is with the line with (*) the file path is wrong. This is the file path that is being retrieved. "'C:\Program Files\Common Files\Microsoft Shared\DevServer\10.0\~\Temp\2012-06-03 01.25.47.jpg'.". The temp folder is located locally inside my project and I want to retrieved the image within that folder. How can I set the file path to my desired location? Or is there another way to convert a file to byte array after retrieving it from a jquery ajax call?
Credits to these articles:
Save and Retrieve Files from SQL Server Database using ASP.NET
Async file upload with jQuery and ASP.NET
Just these 3 lines will do:
int filelength = file.ContentLength;
byte[] imagebytes = new byte[filelength ];
file.InputStream.Read(imagebytes , 0, filelength );
using (var stream = upload.InputStream)
{
// use stream here: using StreamReader, StreamWriter, etc.
}