Develop Reference

unexpected char when parsing json string

I am serializing an object to a stream to store as file and then retrieving and trying to deserialize the object, but get an error parsing. Below is code:
var content = JsonConvert.SerializeObject(data);
var output = new MemoryStream();
var writer = new StreamWriter(output, Encoding.UTF8);
writer.Write(content);
writer.Flush();
//write to some file...
//when reading the file
Stream filestream;
//filestream opens some file stream
byte[] buffer = new byte[4096]
using(MemoryStream ms = new MemoryStream()){
int read;
while((read = filestream.Read(buffer, 0, buffer.Length)) > 0){
ms.Write(buffer, 0, read);
}
var data = Encoding.UTF8.GetString(ms.ToArray());
//encounters error here. I can see that first few chars of the string are question marks.
JsonConvert.DeserializeObject<T>(data);

A couple ideas. Are you getting an exception?
Try encapsulating your write & reads into functions and wrap the memoryStream and streamWriter in using statements as well
using var writer = new StreamWriter(...
Additionally you don't need to set the length of the buffer to 4096, that's probably messing up your encoding. Read your memorystream to an array like so
var buffer = ms.ToArray();
How can I write MemoryStream to byte[]
It looks like you're having an issue copying the memory between the 2 memory streams though. Ideally you would write and read to a file or some other source rather than between 2 memory stream objects but you can copy the contents directly using a copyto
https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.copyto?view=net-6.0
Edit: Adding pseudocode
Assuming your content has been read into
Stream filestream;
using var MemoryStream ms = new MemoryStream();
filestream.CopyTo(ms);
var data = Encoding.UTF8.GetString(ms.ToArray());
return JsonConvert.DeserializeObject<T>(data);

GZipStream does gzipping but ungzipping file end up with "Unexpected end of data"

Does anyone know why I'm getting the "Unexpected end of data" error message when un-gzipping the gzip file?
To verify the bytes data is not corrupted, I use the FooTest4.csv to write to file and was able to opened the file successfully.
Both 'FooTest3.csv.gzand 'FooTest2.csv.gz ran into "Unexpected end of data" when un-gzipping.
public static List<byte> CompressFile(List<byte> parmRawBytes)
{
//Initialize variables...
List<byte> returnModifiedBytes = null;
File.WriteAllBytes(#"X:\FooTest4.csv", parmRawBytes.ToArray());
using (var memoryStream = new MemoryStream())
{
using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress, false))
{
gzipStream.Write(parmRawBytes.ToArray(), 0, parmRawBytes.ToArray().Length);
gzipStream.Flush();
File.WriteAllBytes(#"X:\FooTest3.csv.gz", memoryStream.ToArray());
returnModifiedBytes = memoryStream.ToArray().ToList();
}
}
File.WriteAllBytes(#"X:\FooTest2.csv.gz", returnModifiedBytes.ToArray());
return returnModifiedBytes;
}

GZipStream needs to be closed so it can write some terminating data to the end of the buffer to complete the gzip encoding.
byte[] inputBytes = ...;
using (var compressedStream = new MemoryStream())
{
using (var compressor = new GZipStream(compressedStream, CompressionMode.Compress))
{
compressor.Write(inputBytes, 0, inputBytes.Length);
}
// get bytes after the gzip stream is closed
File.WriteAllBytes(pathToFile, compressedStream.ToArray());
}

Instead of loading the bytes, compressing and saving them you could do do compression and writing at once. Also I don't know why you're using List<Byte> instead of byte[], maybe this could be it.
void CompressFile(string inputPath, string outputPath)
{
Stream readStream = new FileStream(inputPath, Filemode.Open);
Stream writeStream = new FileStream(outputPath, FileMode.Create);
Stream compressionStream = new GZipStream(writeStream. CompressionMode.Compress);
byte[] data = new byte[readStream.Length];
readStream.Read(data, 0, data.Length);
compressionStream.Write(data, 0, data.Length);
readStream.Close();
writeStream.Close();
}
byte[] CompressFile(string inputPath)
{
byte[] data = File.ReadAllBytes(inputPath);
MemoryStream memStream = new MemoryStream(data);
var gzipStream = new GZipStream(memStream, CompressionMode.Compress);
gzipStream.Write(data, 0, data.Length);
gzipStream.Close();
return gzipStream.ToArray();
}
PS: I wrote the code in the text editor, so there might be errors. Also you say the error is on the "unzippiing", why no show us the unzip code?

GZipStream works when writing to FileStream, but not MemoryStream

If compress some json text, and write that it to a file using a FileStream I get the expected results. However, I do not want to write to disk. I simply want to memorystream of the compressed data.
Compression to FileStream:
string json = Resource1.json;
using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
using (FileStream output = File.Create(#"C:\Users\roarker\Desktop\output.json.gz"))
{
using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
{
input.CopyTo(compression);
}
}
Above works. Below, the output memory stream is length 10 and results in an empty .gz file.
string json = Resource1.json;
using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
using (MemoryStream output = new MemoryStream())
{
using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
{
input.CopyTo(compression);
byte[] bytes = output.ToArray();
}
}
EDIT:
Moving output.ToArray() outside the inner using clause seems to work. However, this closes the output stream for most usage. IE:
using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
using (MemoryStream output = new MemoryStream())
{
using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
{
input.CopyTo(compression);
}
WriteToFile(output);
}
where :
public static void WriteToFile(Stream stream)
{
using (FileStream output = File.Create(#"C:\Users\roarker\Desktop\output.json.gz"))
{
stream.CopyTo(output);
}
}
This will fail on stream.CopyTo because the stream has been closed. I know I could make a new Stream from bytes of output.ToArray(), but why is this necessary? why does ToArray() work when the stream is closed?
Final Edit:
Just needed to use the contructor of the GZipStream with the leaveOpen parameter.

You're calling ToArray() before you've closed the GZipStream... that means it hasn't had a chance to flush the final bits of its buffer. This is a common issue for compression an encryption streams, where closing the stream needs to write some final pieces of data. (Even calling Flush() explicitly won't help, for example.)
Just move the ToArray call:
using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
using (MemoryStream output = new MemoryStream())
{
using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
{
input.CopyTo(compression);
}
byte[] bytes = output.ToArray();
// Use bytes
}
(Note that the stream will be disposed when you call ToArray, but that's okay.)

How to write NAudio WaveStream to a Memory Stream?

I have a program that takes in mp3 data in a byte array. It has to convert that mp3 data into wav format and store it in a byte data. I am trying to use NAudio for this purpose. I am using the following code for this purpose.
Stream inputStream = ...;
Stream outputStream = ...;
using (WaveStream waveStream = WaveFormatConversionStream.CreatePcmStream(new Mp3FileReader(inputStream)))
using (WaveFileWriter waveFileWriter = new WaveFileWriter(outputStream, waveStream.WaveFormat))
{
byte[] bytes = new byte[waveStream.Length];
waveStream.Read(bytes, 0, waveStream.Length);
waveFileWriter.WriteData(bytes, 0, bytes.Length);
waveFileWriter.Flush();
}
When I run the above code, all I receive is 0 in the byte array. But if use WaveFileWriter to write the data directly to a file, the file receives the correct data. Any reasons?

Give this a try:
using (WaveStream waveStream = WaveFormatConversionStream.CreatePcmStream(new Mp3FileReader(inputStream)))
using (WaveFileWriter waveFileWriter = new WaveFileWriter(outputStream, waveStream.WaveFormat))
{
byte[] bytes = new byte[waveStream.Length];
waveStream.Position = 0;
waveStream.Read(bytes, 0, waveStream.Length);
waveFileWriter.WriteData(bytes, 0, bytes.Length);
waveFileWriter.Flush();
}

If you are writing to a MemoryStream, you need to be aware that WaveFileWriter will dispose that MemoryStream after you dispose the WaveFileWriter.
Here's a workaround using the IgnoreDisposeStream. (Also note that WaveFormatConversionStream.CreatePcmStream is unnecessary - Mp3FileReader already returns PCM from Read). I also prefer to read in smaller chunks that trying to pass through the whole file.
var path = #"mytestFile.mp3";
var mp3ByteArray = File.ReadAllBytes(path);
var outputStream = new MemoryStream();
using (var mp3Stream = new MemoryStream(mp3ByteArray))
using (var reader = new Mp3FileReader(mp3Stream))
using (var waveFileWriter = new WaveFileWriter(new IgnoreDisposeStream(outputStream),
reader.WaveFormat))
{
byte[] buffer = new byte[reader.WaveFormat.AverageBytesPerSecond];
int read;
while((read = reader.Read(buffer,0, buffer.Length)) > 0)
{
waveFileWriter.Write(buffer, 0, read);
}
}
// outputStream has not yet been disposed so we can get the byte array from it
var wavBytes = outputStream.GetBuffer();
// or we could play it like this
outputStream.Position = 0;
using (var player = new WaveOutEvent())
using (var reader = new WaveFileReader(outputStream))
{
player.Init(reader);
player.Play();
while(player.PlaybackState != PlaybackState.Stopped)
{
Thread.Sleep(1000);
}
}

Create new FileStream out of a byte array

I am attempting to create a new FileStream object from a byte array. I'm sure that made no sense at all so I will try to explain in further detail below.
Tasks I am completing:
1) Reading the source file which was previously compressed
2) Decompressing the data using GZipStream
3) copying the decompressed data into a byte array.
What I would like to change:
1) I would like to be able to use File.ReadAllBytes to read the decompressed data.
2) I would then like to create a new filestream object usingg this byte array.
In short, I want to do this entire operating using byte arrays. One of the parameters for GZipStream is a stream of some sort, so I figured I was stuck using a filestream. But, if some method exists where I can create a new instance of a FileStream from a byte array - then I should be fine.
Here is what I have so far:
FolderBrowserDialog fbd = new FolderBrowserDialog(); // Shows a browser dialog
fbd.ShowDialog();
// Path to directory of files to compress and decompress.
string dirpath = fbd.SelectedPath;
DirectoryInfo di = new DirectoryInfo(dirpath);
foreach (FileInfo fi in di.GetFiles())
{
zip.Program.Decompress(fi);
}
// Get the stream of the source file.
using (FileStream inFile = fi.OpenRead())
{
//Create the decompressed file.
string outfile = #"C:\Decompressed.exe";
{
using (GZipStream Decompress = new GZipStream(inFile,
CompressionMode.Decompress))
{
byte[] b = new byte[blen.Length];
Decompress.Read(b,0,b.Length);
File.WriteAllBytes(outfile, b);
}
}
}
Thanks for any help!
Regards,
Evan

It sounds like you need to use a MemoryStream.

Since you don't know how many bytes you'll be reading from the GZipStream, you can't really allocate an array for it. You need to read it all into a byte array and then use a MemoryStream to decompress.
const int BufferSize = 65536;
byte[] compressedBytes = File.ReadAllBytes("compressedFilename");
// create memory stream
using (var mstrm = new MemoryStream(compressedBytes))
{
using(var inStream = new GzipStream(mstrm, CompressionMode.Decompress))
{
using (var outStream = File.Create("outputfilename"))
{
var buffer = new byte[BufferSize];
int bytesRead;
while ((bytesRead = inStream.Read(buffer, 0, BufferSize)) != 0)
{
outStream.Write(buffer, 0, bytesRead);
}
}
}
}

Here is what I ended up doing. I realize that I did not give sufficient information in my question - and I apologize for that - but I do know the size of the file I need to decompress as I am using it earlier in my program. This buffer is referred to as "blen".
string fi = #"C:\Path To Compressed File";
// Get the stream of the source file.
// using (FileStream inFile = fi.OpenRead())
using (MemoryStream infile1 = new MemoryStream(File.ReadAllBytes(fi)))
{
//Create the decompressed file.
string outfile = #"C:\Decompressed.exe";
{
using (GZipStream Decompress = new GZipStream(infile1,
CompressionMode.Decompress))
{
byte[] b = new byte[blen.Length];
Decompress.Read(b,0,b.Length);
File.WriteAllBytes(outfile, b);
}
}
}