I try to write a List to a FileStream. Since the object was too big, I split the list in evenly distributed chunks, and append it to the FileStream with a DeflateStream to compress the data. This all works fine.
However if I try to do the same to decompress it, it gives an error: 'reached the end of the stream before parsing was completed'. This is the code to decompress:
using (FileStream fs = new FileStream(filePath, FileMode.Open)) {
using (DeflateStream ds = new DeflateStream(fs, CompressionMode.Decompress, true)) {
//Deserialize offerte
BinaryFormatter bf = new BinaryFormatter();
//Check position
while (ds.BaseStream.Position < ds.BaseStream.Length) {
result.AddRange((List<User>)bf.Deserialize(ds));
}
}
}
What I notice is that the first chunk of users is nicely being written to result. However when it should start on the second chunk of users it gives an error right away (It seems before even trying to get the second chunk). What can I do about this or is wrong?
#Edit 10:43 - 16-10-2015 Additional remark
If I skip the DeflateStream and only use FileStream, then it works like a charm.
Compress method (I call this function x times, every chunk once):
using (FileStream fs = new FileStream(filePath, FileMode.Append)) {
using (DeflateStream cs = new DeflateStream(fs, CompressionMode.Compress)) {
//Serialize offerte
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(cs, offerte);
}
}
Related
I'm using this code to write an MP3 MemoryStream to file:
using (var nSpeakStreamAsMp3 = new MemoryStream())
using (var nWavFileReader = new WaveFileReader(nSpeakStream))
using (var nMp3Writer = new LameMP3FileWriter(nSpeakStreamAsMp3, nWavFileReader.WaveFormat, LAMEPreset.STANDARD_FAST))
{
nWavFileReader.CopyTo(nMp3Writer);
string sPath = "C:\\inetpub\\wwwroot\\server\\bin\\mymp3.mp3";
using (FileStream nFile = new FileStream(sPath, FileMode.Create, System.IO.FileAccess.Write))
{
nSpeakStreamAsMp3.CopyTo(nFile);
}
sRet = (String.Concat("data:audio/mpeg;base64,", Convert.ToBase64String(nSpeakStreamAsMp3.ToArray())));
}
return sRet;
For some reason which I don't see, this produces a file of 0 bytes.
However, the MP3 stream is valid and does work. I'm passing it as a Base64String to a website, and I do hear it.
Where might be the error here?
nSpeakStreamAsMp3 is currently positioned at the end of the stream; you need to think like a VCR: be kind, rewind (nSpeakStreamAsMp3.Position = 0;) before you copy the value out again
make sure you flush nMp3Writer; if possible, close nMp3Writer completely
I onced managed to create the bin-file in my project. I changed the primary key from int to Guid and moved code from Main to my class Quote. At the moment I can only add new entries in said file. If I remove it a new file(0 bytes) is created and the stream gets ArgumentException when I try to feed the file dummy-data. I am trying to use an if-loop to handle stream.Lenght == 0.
public static List<Quote> readBinaryToList() //Crashes if binfile is 0 bytes long
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(#"C:\Users\xxxxxx\Desktop\quotes.bin", FileMode.OpenOrCreate, FileAccess.Read, FileShare.Read);
if (stream.Length == 0)
{
Quote q = new Quote(Guid.NewGuid(), "Quote dummy", false);
List<Quote> quoteList = new List<Quote>();
quoteList.Add(q);
var bformatter = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
bformatter.Serialize(stream, quoteList);
bformatter = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
List<Quote> quoteListTmp = (List<Quote>)bformatter.Deserialize(stream);
return quoteList;
}
else
{
List<Quote> quoteList = (List<Quote>)formatter.Deserialize(stream);
stream.Close();
return quoteList;
}
}
As pointed out in previous answers, you must give your file stream write permissions which can be done in its constructor, then you should also set the position of the stream back to 0, you can achieve this by using the stream's Position property.
You are creating a lot of unnecessary objects that don't actually contribute to purpose of the method I have omitted these below. In doing so, setting the streams Position property to 0 is redundant but I've left it in a comment to show how its done.
Some other things to consider: Declare the file stream inside a using statement so that it is disposed when the method comes to an end this means you can omit the manual close in the else statement. Some of your code can be written more tersely, this is just a personal preference but I think it would be best to inline some of your code to remove as much noise as possible. It is also convention in C# to use PascalCase for Methods.
public static List<Quote> ReadBinaryToList(){
using(Stream stream = new FileStream(#"quotes.bin", FileMode.OpenOrCreate, FileAccess.ReadWrite)) {
IFormatter formatter = new BinaryFormatter();
if (stream.Length == 0) {
List<Quote> quoteList = new List<Quote> {new Quote(Guid.NewGuid(), "Quote dummy", false)};
formatter.Serialize(stream, quoteList);
//stream.Position = 0;
return quoteList;
}
else return (List<Quote>)formatter.Deserialize(stream);
}
}
The file is being opened as readonly, serializing to the file will require write permissions.
Stream stream = new FileStream(#"C:\temp\quotes.bin", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.Read);
The stream should also be returned to the beginning before making any attempts to deserialize from it.
stream.Seek(0, SeekOrigin.Begin);
FileStreams have a single "head" where all read and write operations take places. As a new stream is being written, the head is always at the end and any attempt to read from the end will fail. Some streams (e.g. NetworkStream) behave differently and do not allow seeking at all.
Also, the initial position of the FileStream depends on how the file is opened (based on the specified FileMode). The FileMode specified in the question will result in the stream position starting at the beginning of the file, so this is not required in the else block.
And make sure that the Quote class is marked [Serializable]
I have both a BinaryReader and a BinaryWriter open on the same underlying FileStream. When I seek to a location using the BinaryWriter and write a value, it is getting written at the start of the stream instead of the location that was seeked to in the previous line.
My code (simplified):
using (var stream = new FileStream(outputFile, FileMode.Create))
using (var sw = new BinaryWriter(stream))
using (var sr = new BinaryReader(stream))
{
int address;
int value;
//Write some data using sw
//Seek and read a few different places using sr,
//and set address and value
sw.BaseStream.Seek(address, SeekOrigin.Begin);
sw.Write(value);
}
The Position property is showing the correct value in both the FileStream and the BinaryWriter.BaseStream when I inspect them immediately after the Seek, yet it is still writing to the start of the stream.
I also noticed that after performing a seek and read with the BinaryReader, the data that was initially written to the stream gets shifted forward 4 bytes, filling the first 4 bytes with 0.
What could be causing this strange behaviour? Is using two streams at a time valid?
UPDATE
Everything works just fine if I use a MemoryStream instead and then write the data from that to my file. So why would this happening with a FileStream and not a MemoryStream?
I have a file with size 10124, I am adding a byte array, which has length 4 in the beginning of the file.
After that the file size should become 10128, but as I write it to file, the size decreased to 22 bytes. I don't know where is the problem
public void AppendAllBytes(string path, byte[] bytes)
{
var encryptedFile = new FileStream(path, FileMode.Open, FileAccess.Read);
////argument-checking here.
Stream header = new MemoryStream(bytes);
var result = new MemoryStream();
header.CopyTo(result);
encryptedFile.CopyTo(result);
using (var writer = new StreamWriter(#"C:\\Users\\life.monkey\\Desktop\\B\\New folder (2)\\aaaaaaaaaaaaaaaaaaaaaaaaaaa.docx.aef"))
{
writer.Write(result);
}
}
How can I write bytes to the file?
The issue seems to be caused by:
using a StreamWriter to write binary formatted data. The name does not inthuitively suggest this, but the StreamWriter class is suited for writing textual data.
passing an entire stream instead of the actual binary data. To obtain the bytes stored in a MemoryStream, use its convenient ToArray() method.
I suggest you the following code:
public void AppendAllBytes(string path, byte[] bytes)
{
var fileName = #"C:\\Users\\life.monkey\\Desktop\\B\\New folder (2)\\aaaaaaaaaaaaaaaaaaaaaaaaaaa.docx.aef";
using (var encryptedFile = new FileStream(path, FileMode.Open, FileAccess.Read))
using (var writer = new BinaryWriter(File.Open(fileName, FileMode.Append)))
using (var result = new MemoryStream())
{
encryptedFile.CopyTo(result);
result.Flush(); // ensure header is entirely written.
// write header directly, no need to put it in a memory stream
writer.Write(bytes);
writer.Flush(); // ensure the header is written to the result stream.
writer.Write(result.ToArray());
writer.Flush(); // ensure the encryptdFile is written to the result stream.
}
}
The code above uses the BinaryWriter class which is better suited for binary data. It has a Write(byte[] bytes) method overload that is used above to write an entire array to the file. The code uses regular calls to the Flush() method that some may consider not needed, but these guarantee in general, that all the data written prior the call of the Flush() method is persisted within the stream.
I don't understand what I'm doing wrong here. I generate couple of memory streams and in debug-mode I see that they are populated. But when I try to copy MemoryStream to FileStream in order to save the file fileStream is not populated and file is 0bytes long (empty).
Here is my code
if (file.ContentLength > 0)
{
var bytes = ImageUploader.FilestreamToBytes(file); // bytes is populated
using (var inStream = new MemoryStream(bytes)) // inStream is populated
{
using (var outStream = new MemoryStream())
{
using (var imageFactory = new ImageFactory())
{
imageFactory.Load(inStream)
.Resize(new Size(320, 0))
.Format(ImageFormat.Jpeg)
.Quality(70)
.Save(outStream);
}
// outStream is populated here
var fileName = "test.jpg";
using (var fileStream = new FileStream(Server.MapPath("~/content/u/") + fileName, FileMode.CreateNew, FileAccess.ReadWrite))
{
outStream.CopyTo(fileStream); // fileStream is not populated
}
}
}
}
You need to reset the position of the stream before copying.
outStream.Position = 0;
outStream.CopyTo(fileStream);
You used the outStream when saving the file using the imageFactory. That function populated the outStream. While populating the outStream the position is set to the end of the populated area. That is so that when you keep on writing bytes to the steam, it doesn't override existing bytes. But then to read it (for copy purposes) you need to set the position to the start so you can start reading at the start.
If your objective is simply to dump the memory stream to a physical file (e.g. to look at the contents) - it can be done in one move:
System.IO.File.WriteAllBytes(#"C:\\filename", memoryStream.ToArray());
No need to set the stream position first either, since the .ToArray() operation explicitly ignores that, as per #BaconBits comment below https://learn.microsoft.com/en-us/dotnet/api/system.io.memorystream.toarray?view=netframework-4.7.2.
Another alternative to CopyTo is WriteTo.
Advantage:
No need to reset Position.
Usage:
outStream.WriteTo(fileStream);
Function Description:
Writes the entire contents of this memory stream to another stream.