I have a big problem with copying files. When I copy files by File.Copy or FileStream, executing code kill my server. I copied ten files from remote directory to temp dir. The biggest file have around 20MB and cause error
At first I tried File.Copy(), when I run code, server collapsed.
foreach (string file in customerFiles)
{
File.Copy(file, tempOnFile);
}
Same server collapsed happened when I tried this code in foreach:
byte[] buffer;
FileStream fileReader = new FileStream(file, FileMode.Open, FileAccess.Read);
try
{
int bufferSize = (int)fileReader.Length;
buffer = new byte[bufferSize];
int count, sum = 0;
while ((count = fileReader.Read(buffer, sum, bufferSize - sum)) > 0 )
{
sum += count;
}
}
catch(Exception ex)
{
throw ex;
}
finally
{
fileReader.Flush();
fileReader.Close();
}
//Write
string fileOnTemp = pathToTemp + fileName;
FileStream fileWriter = new FileStream(fileOnTemp, FileMode.CreateNew, FileAccess.Write);
try
{
fileWriter.Write(buffer, 0, buffer.Length);
}
catch(Exception ex)
{
throw ex;
}
finally
{
fileWriter.Flush();
fileWriter.Close();
}
Please Help! Thank you,
Michal
Related
Problem:
In installed Office 2010 computer, my app have to copy an empty excel file (file A) to new excel file (file B) and use OpenXML library (V2.5) to execute some action, finally saved to hard disk. After that I open file B and just add a litle bit data (for example: 1) to it and save and close it.
when I reopen file B, excel thrown an error: Excel found unreadable content in ' file B' do you want to recover the contents of this workbook... and I can not open it.
Below is my code:
static void Main(string[] args)
{
ExportDataSet(#"C:\A.xlsx",#"C:\");
}
public static void Copy(String oldPath, String newPath)
{
FileStream input = null;
FileStream output = null;
try
{
input = new FileStream(oldPath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
output = new FileStream(newPath, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
var buffer = new byte[32768];
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, read);
}
}
catch (Exception e)
{
}
finally
{
if (input != null)
{
input.Close();
input.Dispose();
}
if (output != null)
{
output.Close();
output.Dispose();
}
}
}
public static string ExportDataSet(string filePath, string path, int startRow = 10)
{
var pathToSave = path;
if (!Directory.Exists(pathToSave))
Directory.CreateDirectory(pathToSave);
var filename = pathToSave + Guid.NewGuid() + Path.GetExtension(filePath);
Copy(filePath, filename);
var fs = File.Open(filename, FileMode.Open);
{
using (var myWorkbook = SpreadsheetDocument.Open(fs, true))
{
var workbookPart = myWorkbook.WorkbookPart;
var Sheets = myWorkbook.WorkbookPart.Workbook.GetFirstChild<Sheets>().Elements<Sheet>();
var relationshipId = Sheets.First().Id.Value;
var worksheetPart = (WorksheetPart)myWorkbook.WorkbookPart.GetPartById(relationshipId);
var sheetData = worksheetPart.Worksheet.GetFirstChild<SheetData>();
workbookPart.Workbook.Save();
//workbookPart.Workbook.CalculationProperties = new CalculationProperties() { FullCalculationOnLoad = true };
}
fs.Close();
fs.Dispose();
return filename;
}
}
I think the OpenXML library has something wrong.
Do you have any ideas? please share to me, thank you so much.
Remarks:
1. the computer use Office 2010 to open Excel file
2. the file format is Excel workbook (.xlsx)
3. if the computer installed office with later version (2013, 2016), the problem was not appeared.
Your buffer reading and writting logic is wrong. The second parameter is where it starts to read or write and you are passing it a zero value, so second iteration of the while is overwritting the content written in the first iteration thus you are getting corrupted data if the files are greater than your buffer size.
Your code should be similar to this:
var buffer = new byte[32768];
int totalRead = 0; // Variable to track where to write in the next while iteration
int read;
while ((read = input.Read(buffer, totalRead, buffer.Length)) > 0)
{
output.Write(buffer, totalRead, read);
totalRead += read; // Add to totalRead the amount written so next iteration does append the content at the end.
}
I have a service running on a server that zip files and I notice that each day the memory consumed increases, when I deployed it on the server it was consuming 3.6Mb, today, 3 months later it was consuming 180Mb.
This is part of the code that I'm using:
for (i = 0; i < files.Count; i++)
{
try
{
if (File.Exists(#dir + zipToUpdate) && new FileInfo(#dir + zipToUpdate).Length < 104857600)
{
using (FileStream zipToOpen = new FileStream(#dir + zipToUpdate, FileMode.Open))
{
using (ZipArchive archive = new ZipArchive(zipToOpen, ZipArchiveMode.Update, false))
{
if (File.GetCreationTime(#dir + files.ElementAt(i)).AddHours(FileAge) < DateTime.Now)
{
ZipArchiveEntry fileEntry = archive.CreateEntry(files.ElementAt(i));
using (BinaryWriter writer = new BinaryWriter(fileEntry.Open()))
{
using (FileStream sr = new FileStream(#dir + files.ElementAt(i), FileMode.Open, FileAccess.Read))
{
byte[] block = new byte[32768];
int bytesRead = 0;
while ((bytesRead = sr.Read(block, 0, block.Length)) > 0)
{
writer.Write(block, 0, bytesRead);
block = new byte[32768];
}
}
}
File.Delete(#dir + files.ElementAt(i));
}
}
}
}
else
{
createZip(files.GetRange(i, files.Count-i), dir + "\\", getZipName(dir, zipToUpdate));
return;
}
}
catch (Exception ex)
{
rootlog.Error(string.Format("Erro Run - updateZip: {0}", ex.Message));
}
}
The creation of the zip or the update are similar so there is no point in paste both codes.
I do a recursive call of this for the folders inside and the service runs once each hour.
So, my question is if all these streams is what is making my memory usage increase month after month or if it can be something else.
The using statement takes care of closing the IDisposable object that it opens. This is not the source of the potential memory leak you're observing.
I have a 453MB XML file which I'm trying to compress to a ZIP using SharpZipLib.
Below is the code I'm using to create the zip, but it's causing an OutOfMemoryException. This code successfully compresses a file of 428MB.
Any idea why the exception is happening, as I can't see why, as my system has plenty of memory available.
public void CompressFiles(List<string> pathnames, string zipPathname)
{
try
{
using (FileStream stream = new FileStream(zipPathname, FileMode.Create, FileAccess.Write, FileShare.None))
{
using (ZipOutputStream stream2 = new ZipOutputStream(stream))
{
foreach (string str in pathnames)
{
FileStream stream3 = new FileStream(str, FileMode.Open, FileAccess.Read, FileShare.Read);
byte[] buffer = new byte[stream3.Length];
try
{
if (stream3.Read(buffer, 0, buffer.Length) != buffer.Length)
{
throw new Exception(string.Format("Error reading '{0}'.", str));
}
}
finally
{
stream3.Close();
}
ZipEntry entry = new ZipEntry(Path.GetFileName(str));
stream2.PutNextEntry(entry);
stream2.Write(buffer, 0, buffer.Length);
}
stream2.Finish();
}
}
}
catch (Exception)
{
File.Delete(zipPathname);
throw;
}
}
You're trying to create a buffer as big as the file. Instead, make the buffer a fixed size, read some bytes into it, and write the number of read bytes into the zip file.
Here's your code with a buffer of 4096 bytes (and some cleanup):
public static void CompressFiles(List<string> pathnames, string zipPathname)
{
const int BufferSize = 4096;
byte[] buffer = new byte[BufferSize];
try
{
using (FileStream stream = new FileStream(zipPathname,
FileMode.Create, FileAccess.Write, FileShare.None))
using (ZipOutputStream stream2 = new ZipOutputStream(stream))
{
foreach (string str in pathnames)
{
using (FileStream stream3 = new FileStream(str,
FileMode.Open, FileAccess.Read, FileShare.Read))
{
ZipEntry entry = new ZipEntry(Path.GetFileName(str));
stream2.PutNextEntry(entry);
int read;
while ((read = stream3.Read(buffer, 0, buffer.Length)) > 0)
{
stream2.Write(buffer, 0, read);
}
}
}
stream2.Finish();
}
}
catch (Exception)
{
File.Delete(zipPathname);
throw;
}
}
Especially note this block:
const int BufferSize = 4096;
byte[] buffer = new byte[BufferSize];
// ...
int read;
while ((read = stream3.Read(buffer, 0, buffer.Length)) > 0)
{
stream2.Write(buffer, 0, read);
}
This reads bytes into buffer. When there are no more bytes, the Read() method returns 0, so that's when we stop. When Read() succeeds, we can be sure there is some data in the buffer but we don't know how many bytes. The whole buffer might be filled, or just a small portion of it. Therefore, we use the number of read bytes read to determine how many bytes to write to the ZipOutputStream.
That block of code, by the way, can be replaced by a simple statement that was added to .Net 4.0, which does exactly the same:
stream3.CopyTo(stream2);
So, your code could become:
public static void CompressFiles(List<string> pathnames, string zipPathname)
{
try
{
using (FileStream stream = new FileStream(zipPathname,
FileMode.Create, FileAccess.Write, FileShare.None))
using (ZipOutputStream stream2 = new ZipOutputStream(stream))
{
foreach (string str in pathnames)
{
using (FileStream stream3 = new FileStream(str,
FileMode.Open, FileAccess.Read, FileShare.Read))
{
ZipEntry entry = new ZipEntry(Path.GetFileName(str));
stream2.PutNextEntry(entry);
stream3.CopyTo(stream2);
}
}
stream2.Finish();
}
}
catch (Exception)
{
File.Delete(zipPathname);
throw;
}
}
And now you know why you got the error, and how to use buffers.
You're allocating a lot of memory for no good reason, and I bet you have a 32-bit process. 32-bit processes can only allocate up to 2GB of virtual memory in normal conditions, and the library surely allocates memory too.
Anyway, several things are wrong here:
byte[] buffer = new byte[stream3.Length];
Why? You don't need to store the whole thing in memory to process it.
if (stream3.Read(buffer, 0, buffer.Length) != buffer.Length)
This one is nasty. Stream.Read is explicitly allowed to return less bytes than what you asked for, and this is still a valid result. When reading a stream into a buffer you have to call Read repeatedly until the buffer is filled or the end of the stream is reached.
Your variables should have more meaningful names. You can easily get lost with these stream2, stream3 etc.
A simple solution would be:
using (var zipFileStream = new FileStream(zipPathname, FileMode.Create, FileAccess.Write, FileShare.None))
using (ZipOutputStream zipStream = new ZipOutputStream(zipFileStream))
{
foreach (string str in pathnames)
{
using(var itemStream = new FileStream(str, FileMode.Open, FileAccess.Read, FileShare.Read))
{
var entry = new ZipEntry(Path.GetFileName(str));
zipStream.PutNextEntry(entry);
itemStream.CopyTo(zipStream);
}
}
zipStream.Finish();
}
I want to compress a Excel file to .zip or .cap extension. The code Used to do that it is compressing the file but that zip file can't be unzip. while unzip that i am getting the error file file corrupted or can't be unzip.
The code I am using:
static public bool CompressFile(string file, string outputFile)
{
try
{
using (var inFile = File.OpenRead(file))
{
using (var outFile = File.Create(outputFile))
{
using (var compress = new GZipStream(outFile, CompressionMode.Compress, false))
{
byte[] buffer = new byte[inFile.Length];
int read = inFile.Read(buffer, 0, buffer.Length);
while (read > 0)
{
compress.Write(buffer, 0, read);
read = inFile.Read(buffer, 0, buffer.Length);
}
}
}
}
return true;
}
catch (IOException ex)
{
MessageBox.Show(string.Format("Error compressing file: {0}", ex.Message));
return false;
}
}
Even i go some link to get the proper solution. But nothing is workout.I need some suggestion to get the proper solution. Any answer please.
This code uses the SharpZipLib library and will compress files that can be uncompressed no problems
private void Zip()
{
string output = #"C:\TEMP\test.zip";
string input = #"C:\TEMP\test.xlsx";
using (var zipStream = new ZipOutputStream(System.IO.File.Create(output)))
{
zipStream.SetLevel(9);
var buffer = new byte[4096];
var entry = new ZipEntry(Path.GetFileName(input));
zipStream.PutNextEntry(entry);
using (FileStream fs = System.IO.File.OpenRead(input))
{
int sourceBytes;
do
{
sourceBytes = fs.Read(buffer, 0, buffer.Length);
zipStream.Write(buffer, 0, sourceBytes);
} while (sourceBytes > 0);
}
zipStream.Finish();
zipStream.Close();
}
}
The program is meant to set the file path and the idea is that when the data is set, it should use this function:
public void SendFile(String fileName, long fileSize, NetworkStream io)
{
SendFileNameToServer();
SendFileSizeToServer();
byte[] fileData;
try
{
if (!File.Exists(fileName))
{
throw new FileNotFoundException("File does not exist!");
}
FileStream openFileStream = new FileStream(fileName, FileMode.Open, FileAccess.Read);
BinaryReader bReader = new BinaryReader(openFileStream);
Int32 remainingSize = Convert.ToInt32(_fileSize);
do
{
fileData = bReader.ReadBytes(BUFSIZE);
io.Write(fileData, 0, BUFSIZE);
remainingSize -= BUFSIZE;
} while (remainingSize > BUFSIZE);
do
{
fileData = bReader.ReadBytes(remainingSize);
io.Write(fileData, 0, remainingSize);
remainingSize -= remainingSize;
} while (remainingSize > 0);
bReader.Close();
openFileStream.Flush();
openFileStream.Close();
io.Flush();
io.Close();
}
catch (Exception)
{
throw new Exception();
}
}
to send the file given in the file path to a server-side program which receives the file data.
The problem is that when it gets to the line FileStream openFileStream = new FileStream(fileName, FileMode.Open, FileAccess.Read); it tells me that the file is not found. The exception it gives is Exception:Thrown: "The process cannot access the file 'D:\StepMania\Songs\Fragma\You Are Alive\Green.avi' because it is being used by another process." (System.IO.IOException)
A System.IO.IOException was thrown: "The process cannot access the file 'D:\*FilePath*\Green.avi' because it is being used by another process."
Time: 04-05-2013 21:11:39
Thread:Main Thread[5532] but I can't think of any process that would use this file when StepMania is not running.
I know that the file is there because I check the file path and it's there as it should be. It works just fine if the file is in the exact same folder as the program but other than that, I can't seem to find the solution to this problem. Does anyone have any ideas as to what could be wrong?
If you need any more of my code, please tell me.
Edit:
My server uses this code to receive the file:
public void ReceiveFile(String fileName, NetworkStream io)
{
// TO DO Din egen kode
byte[] fileData = new byte[BUFSIZE];
FileStream writeFileStream = new FileStream(fileName, FileMode.Create);
BinaryWriter bWrite = new BinaryWriter(writeFileStream);
int bytesRead = 0;
long remainingSize = Convert.ToInt32(_fileSize);
do
{
Console.WriteLine("Remaining number of bytes: {0}", remainingSize);
bytesRead = io.Read(fileData, 0, BUFSIZE); // Read max 1000 bytes from server via socket (actual value is placed in "bytesRead"
bWrite.Write(fileData, 0, bytesRead); // write the received bytes into file. the number of received bytes is placed in "bytesRead"
remainingSize -= bytesRead;
}
while (remainingSize > 0);
writeFileStream.Flush();
writeFileStream.Close();
bWrite.Close();
}
Ok, I found the problem: My server-side program interefered with my client-side program. Here's the fixed code for the SendFile code of my client program:
public void SendFile(String fileName, long fileSize, NetworkStream io)
{
SendFileNameToServer();
SendFileSizeToServer();
byte[] fileData;
try
{
FileStream openFileStream = File.OpenRead(fileName);
BinaryReader bReader = new BinaryReader(openFileStream);
Int32 remainingSize = Convert.ToInt32(_fileSize);
do
{
fileData = bReader.ReadBytes(BUFSIZE);
io.Write(fileData, 0, BUFSIZE);
remainingSize -= BUFSIZE;
} while (remainingSize > BUFSIZE);
do
{
fileData = bReader.ReadBytes(remainingSize);
io.Write(fileData, 0, remainingSize);
remainingSize -= remainingSize;
} while (remainingSize > 0);
openFileStream.Flush();
bReader.Close();
openFileStream.Close();
io.Flush();
io.Close();
}
catch (Exception)
{
throw new Exception();
}
}
Here's the ReceiveFile code for my server:
public void ReceiveFile(String fileName, NetworkStream io)
{
// TO DO Din egen kode
byte[] fileData = new byte[BUFSIZE];
FileStream writeFileStream = new FileStream(LIB.extractFileName(fileName), FileMode.Create);
BinaryWriter bWrite = new BinaryWriter(writeFileStream);
int bytesRead = 0;
long remainingSize = Convert.ToInt32(_fileSize);
do
{
Console.WriteLine("Remaining number of bytes: {0}", remainingSize);
bytesRead = io.Read(fileData, 0, BUFSIZE); // Read max 1000 bytes from server via socket (actual value is placed in "bytesRead"
bWrite.Write(fileData, 0, bytesRead); // write the received bytes into file. the number of received bytes is placed in "bytesRead"
remainingSize -= bytesRead;
}
while (remainingSize > 0);
writeFileStream.Flush();
writeFileStream.Close();
bWrite.Close();
}
Again, thank you everyone who answered my post!