I'm working with GZipStream at the moment using .net 3.5.
I have two methods listed below. As input file I use text file which consists of chars 's'. Size of the file is 2MB. This code works fine if I use .net 4.5 but with .net 3.5 after compress and decompress I get file of size 435KB which of course isn't the same with source file.
If I try to decompress file via WinRAR it is also looks good (the same with source file).
If I try decompress file using GZipStream from .net4.5 (file compressed via GZipStream from .net 3.5) the result is bad.
UPD:
In general I really need to read the file as several separate gzip chunks, in this case all the bytes of copressed files are read at one call of the Read() method so I still don't understand why decompressing doesn't works.
public void CompressFile()
{
string fileIn = #"D:\sin2.txt";
string fileOut = #"D:\sin2.txt.pgz";
using (var fout = File.Create(fileOut))
{
using (var fin = File.OpenRead(fileIn))
{
using (var zip = new GZipStream(fout, CompressionMode.Compress))
{
var buffer = new byte[1024 * 1024 * 10];
int n = fin.Read(buffer, 0, buffer.Length);
zip.Write(buffer, 0, n);
}
}
}
}
public void DecompressFile()
{
string fileIn = #"D:\sin2.txt.pgz";
string fileOut = #"D:\sin2.1.txt";
using (var fsout = File.Create(fileOut))
{
using (var fsIn = File.OpenRead(fileIn))
{
var buffer = new byte[1024 * 1024 * 10];
int n;
while ((n = fsIn.Read(buffer, 0, buffer.Length)) > 0)
{
using (var ms = new MemoryStream(buffer, 0, n))
{
using (var zip = new GZipStream(ms, CompressionMode.Decompress))
{
int nRead = zip.Read(buffer, 0, buffer.Length);
fsout.Write(buffer, 0, nRead);
}
}
}
}
}
}
You're trying to decompress each "chunk" as if it's a separate gzip file. Don't do that - just read from the GZipStream in a loop:
using (var fsout = File.Create(fileOut))
{
using (var fsIn = File.OpenRead(fileIn))
{
using (var zip = new GZipStream(fsIn, CompressionMode.Decompress))
{
var buffer = new byte[1024 * 32];
int bytesRead;
while ((bytesRead = zip.Read(buffer, 0, buffer.Length)) > 0)
{
fsout.Write(buffer, 0, bytesRead);
}
}
}
}
Note that your compression code should look similar, reading in a loop rather than assuming a single call to Read will read all the data.
(Personally I'd skip fsIn, and just use new GZipStream(File.OpenRead(fileIn)) but that's just a personal preference.)
First, as #Jon Skeet mentioned, you are not using Stream.Read method correctly. It doesn't matter if your buffer is big enough or not, the stream is allowed to return less bytes than requested, with zero indicating no more, so reading from stream should always be performed in a loop.
However the main problem in your decompress code is the way you share the buffer. Your read the input into a buffer, than wrap it in a MemoryStream (note that the constructor used does not make a copy of the passed array, but actually sets it as it's internal buffer), and then you try to read and write to that buffer at the same time. Taking into account that decompressing writes data "faster" than reading, it's surprising that your code works at all.
The correct implementation is quite simple
static void CompressFile()
{
string fileIn = #"D:\sin2.txt";
string fileOut = #"D:\sin2.txt.pgz";
using (var input = File.OpenRead(fileIn))
using (var output = new GZipStream(File.Create(fileOut), CompressionMode.Compress))
Write(input, output);
}
static void DecompressFile()
{
string fileIn = #"D:\sin2.txt.pgz";
string fileOut = #"D:\sin2.1.txt";
using (var input = new GZipStream(File.OpenRead(fileIn), CompressionMode.Decompress))
using (var output = File.Create(fileOut))
Write(input, output);
}
static void Write(Stream input, Stream output, int bufferSize = 10 * 1024 * 1024)
{
var buffer = new byte[bufferSize];
for (int readCount; (readCount = input.Read(buffer, 0, buffer.Length)) > 0;)
output.Write(buffer, 0, readCount);
}
Related
I make an archiver with block-by-block reading and file compression. I put the compressed block in FileStream.
I am reading the 5 mb block. The problem is that if I compress a pic of 8 mb, then when I pull it out of the resulting archive, its sum-hash does not match the original and it opens pic halfway, and the size is the same... I don’t know what to try. I ask for help.
Read chunk void:
private byte[] ReadChunk(int chunkId)
{
using (var inFile = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.Read))
{
long filePosition = chunkId * chunkDataSize;
int bytesRead;
if (inFile.Length - filePosition <= chunkDataSize)
{
bytesRead = (int)(inFile.Length - filePosition);
}
else
{
bytesRead = chunkDataSize;
}
var lastBuffer = new byte[bytesRead];
inFile.Read(lastBuffer, 0, bytesRead);
return lastBuffer;
}
}
Compress and write void:
private void CompressBlock(byte[] bytesTo)
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream gs = new GZipStream(ms, CompressionMode.Compress))
{
gs.Write(bytesTo, 0, bytesTo.Length);
}
byte[] compressedData = ms.ToArray();
using (var outFile = new FileStream(resultFile, FileMode.Append))
{
BitConverter.GetBytes(compressedData.Length).CopyTo(compressedData, 4);
outFile.Write(compressedData, 0, compressedData.Length);
}
}
}
I was writing a light-weight proxy in c#. When I was decoding the gzip contentEncoding I noted that if I use a small buffer-size(4096) the stream is decoded partially depending the size of the input. Is it a bug in my code or something which is needed to make it work? I set the buffer to 10 MB, and it works okay but defeats my purpose of writing a light-weight proxy.
response = webEx.Response as HttpWebResponse;
Stream input = response.GetResponseStream();
//some other operations on response header
//calling DecompressGzip here
private static string DecompressGzip(Stream input, Encoding e)
{
StringBuilder sb = new StringBuilder();
using (Ionic.Zlib.GZipStream decompressor = new Ionic.Zlib.GZipStream(input, Ionic.Zlib.CompressionMode.Decompress))
{
// works okay for [1024*1024*8];
byte[] buffer = new byte[4096];
int n = 0;
do
{
n = decompressor.Read(buffer, 0, buffer.Length);
if (n > 0)
{
sb.Append(e.GetString(buffer));
}
} while (n > 0);
}
return sb.ToString();
}
Actually, I figured it out. I guess using the string builder causes the problem; instead, I used a memory stream and it works well.
private static string DecompressGzip(Stream input, Encoding e)
{
using (Ionic.Zlib.GZipStream decompressor = new Ionic.Zlib.GZipStream(input, Ionic.Zlib.CompressionMode.Decompress))
{
int read = 0;
var buffer = new byte[4096];
using (MemoryStream output = new MemoryStream())
{
while ((read = decompressor.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, read);
}
return e.GetString(output.ToArray());
}
}
}
When I use the same GZipStream to compress file blocks in loop the result file compress successfully:
public static void Compress1(string fi)
{
using (FileStream inFile = File.Open(fi,FileMode.Open,FileAccess.Read,FileShare.Read))
{
using (FileStream outFile = File.Create(fi + ".gz"))
{
using (GZipStream Compress = new GZipStream(outFile,
CompressionMode.Compress))
{
byte[] buffer = new byte[6315120];
int numRead;
while ((numRead = inFile.Read(buffer, 0, buffer.Length)) != 0)
{
Compress.Write(buffer, 0, numRead);
}
}
}
}
}
But when I compress file blocks separately in different streams the result file corrupts:
public static void Compress2(string fi, int offset)
{
using (FileStream inFile = File.Open(fi,FileMode.Open))
{
using (FileStream outFile = File.OpenOrCreate(fi + ".gz"))
{
using (GZipStream Compress = new GZipStream(outFile,
CompressionMode.Compress))
{
// Copy the source file into the compression stream.
byte[] buffer = new byte[6315120];
int numRead=-1;
inFile.Seek(offset,SeekOrigin.Begin);
numRead = inFile.Read(buffer, 0, buffer.Length);
Compress.Write(buffer, 0, numRead);
}
}
}
}
In these examples I have a file with size = 12630240. And devide it into 2 blocks, size of each block = 6315120 (buffer size). So, the first block compress correctly in both methods, but the second block in second method compress different from the first method. What I missed?
What is happening is you are creating to different files as each GZipStream has its one headers
by dividing what you are doing is creating to different GZ files and if you write the two to the same file it is a corrupt file.
I have a program that takes in mp3 data in a byte array. It has to convert that mp3 data into wav format and store it in a byte data. I am trying to use NAudio for this purpose. I am using the following code for this purpose.
Stream inputStream = ...;
Stream outputStream = ...;
using (WaveStream waveStream = WaveFormatConversionStream.CreatePcmStream(new Mp3FileReader(inputStream)))
using (WaveFileWriter waveFileWriter = new WaveFileWriter(outputStream, waveStream.WaveFormat))
{
byte[] bytes = new byte[waveStream.Length];
waveStream.Read(bytes, 0, waveStream.Length);
waveFileWriter.WriteData(bytes, 0, bytes.Length);
waveFileWriter.Flush();
}
When I run the above code, all I receive is 0 in the byte array. But if use WaveFileWriter to write the data directly to a file, the file receives the correct data. Any reasons?
Give this a try:
using (WaveStream waveStream = WaveFormatConversionStream.CreatePcmStream(new Mp3FileReader(inputStream)))
using (WaveFileWriter waveFileWriter = new WaveFileWriter(outputStream, waveStream.WaveFormat))
{
byte[] bytes = new byte[waveStream.Length];
waveStream.Position = 0;
waveStream.Read(bytes, 0, waveStream.Length);
waveFileWriter.WriteData(bytes, 0, bytes.Length);
waveFileWriter.Flush();
}
If you are writing to a MemoryStream, you need to be aware that WaveFileWriter will dispose that MemoryStream after you dispose the WaveFileWriter.
Here's a workaround using the IgnoreDisposeStream. (Also note that WaveFormatConversionStream.CreatePcmStream is unnecessary - Mp3FileReader already returns PCM from Read). I also prefer to read in smaller chunks that trying to pass through the whole file.
var path = #"mytestFile.mp3";
var mp3ByteArray = File.ReadAllBytes(path);
var outputStream = new MemoryStream();
using (var mp3Stream = new MemoryStream(mp3ByteArray))
using (var reader = new Mp3FileReader(mp3Stream))
using (var waveFileWriter = new WaveFileWriter(new IgnoreDisposeStream(outputStream),
reader.WaveFormat))
{
byte[] buffer = new byte[reader.WaveFormat.AverageBytesPerSecond];
int read;
while((read = reader.Read(buffer,0, buffer.Length)) > 0)
{
waveFileWriter.Write(buffer, 0, read);
}
}
// outputStream has not yet been disposed so we can get the byte array from it
var wavBytes = outputStream.GetBuffer();
// or we could play it like this
outputStream.Position = 0;
using (var player = new WaveOutEvent())
using (var reader = new WaveFileReader(outputStream))
{
player.Init(reader);
player.Play();
while(player.PlaybackState != PlaybackState.Stopped)
{
Thread.Sleep(1000);
}
}
I need to write contents of a file to another file using File.OpenRead and File.OpenWrite methods. I am unable to figure out how to do it.
How can i modify the following code to work for me.
using (FileStream stream = File.OpenRead("C:\\file1.txt"))
using (FileStream writeStream = File.OpenWrite("D:\\file2.txt"))
{
BinaryReader reader = new BinaryReader(stream);
BinaryWriter writer = new BinaryWriter(writeStream);
writer.Write(reader.ReadBytes(stream.Length));
}
using (FileStream stream = File.OpenRead("C:\\file1.txt"))
using (FileStream writeStream = File.OpenWrite("D:\\file2.txt"))
{
BinaryReader reader = new BinaryReader(stream);
BinaryWriter writer = new BinaryWriter(writeStream);
// create a buffer to hold the bytes
byte[] buffer = new Byte[1024];
int bytesRead;
// while the read method returns bytes
// keep writing them to the output stream
while ((bytesRead =
stream.Read(buffer, 0, 1024)) > 0)
{
writeStream.Write(buffer, 0, bytesRead);
}
}
Just wonder why not to use this:
File.Copy("C:\\file1.txt", "D:\\file2.txt");
You should be using File.Copy unless you want to append to the second file.
If you want to append you can still use the File class.
string content = File.ReadAllText("C:\\file1.txt");
File.AppendAllText("D:\\file2.txt",content);
This works for file with small size as entire file in loaded into the memory.
Try something along these lines:
using (FileStream input = File.OpenRead(pathToInputFile),
output = File.OpenWrite(pathToOutputFile))
{
int read = -1;
byte[] buffer = new byte[4096];
while (read != 0)
{
read = input.Read(buffer, 0, buffer.Length);
output.Write(buffer, 0, read);
}
}
Note that this is somewhat 'skeletal' and you should amend as required for your application of it.
Is it necessary to us FileStream? Because you can do this very easily with simple File Class like;
using System.IO;
string FileContent = File.ReadAllText(FilePathWhoseTextYouWantToCopy);
File.WriteAllText(FilePathToWhomYouWantToPasteTheText,FileContent);
using (var inputStream = File.OpenRead(#"C:\file1.txt"))
{
using (var outputStream = File.OpenWrite(#"D:\file2.txt"))
{
int bufferLength = 128;
byte[] buffer = new byte[bufferLength];
int bytesRead = 0;
do
{
bytesRead = inputStream.Read(buffer, 0, bufferLength);
outputStream.Write(buffer, 0, bytesRead);
}
while (bytesRead != 0);
}
}
Use FileStream class, from System.IO.
[ComVisibleAttribute(true)]
public class FileStream : Stream
Have you checked that the reader is reading all the data? This MSDN page has an example that checks all the data is read:
byte[] verifyArray = binReader.ReadBytes(arrayLength);
if(verifyArray.Length != arrayLength)
{
Console.WriteLine("Error reading the data.");
return;
}
The other alternative is that you probably need to Flush the output buffer:
writer.Flush();
If you are not keen at using Read/Write function of File , you can better try using Copy functionality
Easiest will be :
File.Copy(source_file_name, destination_file_name, true)
true--> for overwriting existing file,without "true" it will create a new file.But if the file already exists it will throw exception without "true" argument.