Develop Reference

unexpected char when parsing json string

I am serializing an object to a stream to store as file and then retrieving and trying to deserialize the object, but get an error parsing. Below is code:
var content = JsonConvert.SerializeObject(data);
var output = new MemoryStream();
var writer = new StreamWriter(output, Encoding.UTF8);
writer.Write(content);
writer.Flush();
//write to some file...
//when reading the file
Stream filestream;
//filestream opens some file stream
byte[] buffer = new byte[4096]
using(MemoryStream ms = new MemoryStream()){
int read;
while((read = filestream.Read(buffer, 0, buffer.Length)) > 0){
ms.Write(buffer, 0, read);
}
var data = Encoding.UTF8.GetString(ms.ToArray());
//encounters error here. I can see that first few chars of the string are question marks.
JsonConvert.DeserializeObject<T>(data);

A couple ideas. Are you getting an exception?
Try encapsulating your write & reads into functions and wrap the memoryStream and streamWriter in using statements as well
using var writer = new StreamWriter(...
Additionally you don't need to set the length of the buffer to 4096, that's probably messing up your encoding. Read your memorystream to an array like so
var buffer = ms.ToArray();
How can I write MemoryStream to byte[]
It looks like you're having an issue copying the memory between the 2 memory streams though. Ideally you would write and read to a file or some other source rather than between 2 memory stream objects but you can copy the contents directly using a copyto
https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.copyto?view=net-6.0
Edit: Adding pseudocode
Assuming your content has been read into
Stream filestream;
using var MemoryStream ms = new MemoryStream();
filestream.CopyTo(ms);
var data = Encoding.UTF8.GetString(ms.ToArray());
return JsonConvert.DeserializeObject<T>(data);

I am getting a "system.runtime.serialization.serializationexception" when using the binary formatter

private byte[] ToByteArray(Message msg)
{
if(msg == null)
{
return null;
}
BinaryFormatter bf = new BinaryFormatter();
using(MemoryStream ms = new MemoryStream())
{
bf.Serialize(ms, msg);
return ms.ToArray();
}
}
private Message ToMessageObject(Byte[] bytes)
{
if(bytes == null)
{
return null;
}
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream ms = new MemoryStream(bytes))
{
ms.Write(bytes, 0, bytes.Length);
ms.Position = 0;
Message msg = (Message)bf.Deserialize(ms);
return msg;
}
}
I am using the two above methods for serialization and deserialization, but it keeps on throwing the error during deserializaion.
system.runtime.serialization.serializationexception: 'end of stream encountered before parsing was completed.'
My message class has the "Serializable" attribute.
One of the things I noticed is when the text is small, like two words, it deserializes fine but when it contains a lot of characters, close to 100, that's when I get that error. I have been checking other solutions()frome here and other places to this problem but none seems to work for me.

From the comments, I was able to figure out that the problem was caused by the initialization of the byte array,
var responseStream = client.GetStream()
var bytes = new byte[1024];
if (responseStream.DataAvailable)
{
await responseStream.ReadAsync(bytes, 0, bytes.Length);
var responseMessage =ToMessageObject(bytes);
messages.Add(responseMessage);
}
Initially, I set the bytes array length to 256. The error was thrown anytime I had to deserialize a byte array of length greater than 256. So, I am wondering if it's possible to use a dynamic array when reading from a stream.

GZIPStream Compression Always Returns 10 Bytes

I'm trying to compress some text in my UWP application. I created this method to make it easier later on:
public static byte[] Compress(this string s)
{
var b = Encoding.UTF8.GetBytes(s);
using (MemoryStream ms = new MemoryStream())
using (GZipStream zipStream = new GZipStream(ms, CompressionMode.Compress))
{
zipStream.Write(b, 0, b.Length);
zipStream.Flush(); //Doesn't seem like Close() is available in UWP, so I changed it to Flush(). Is this the problem?
return ms.ToArray();
}
}
But unfortunately this always returns 10 bytes, no matter what the input text is. Is it because I don't use .Close() on the GZipStream?

You are returning the byte data too early.
The Close() method is replaced by the Dispose() method. So the GZIP stream will be written only when disposed so after you leave the using(GZipStream) {} block.
public static byte[] Compress(string s)
{
var b = Encoding.UTF8.GetBytes(s);
var ms = new MemoryStream();
using (GZipStream zipStream = new GZipStream(ms, CompressionMode.Compress))
{
zipStream.Write(b, 0, b.Length);
zipStream.Flush(); //Doesn't seem like Close() is available in UWP, so I changed it to Flush(). Is this the problem?
}
// we create the data array here once the GZIP stream has been disposed
var data = ms.ToArray();
ms.Dispose();
return data;
}

MemoryStream disables reading when returned

In my program, I am basically reading in a file, doing some processing to it, and then passing it back to the main program as a memorystream, which will be handled by a streamreader. This will all be handled by a class beside my main.
The problem is, when I return the memory stream from my method in another class, the "canread" variable is set to false, and thus causes the streamreader initialization to fail.
Below is an example of the problem happening (though in here I'm writing to the memorystream in the other class, but it still causes the same error when i pass it back.
In the class named "Otherclass":
public static MemoryStream ImportantStreamManipulator()
{
MemoryStream MemStream = new MemoryStream();
StreamWriter writer = new StreamWriter(MemStream);
using (writer)
{
//Code that writes stuff to the memorystream via streamwriter
return MemStream;
}
}
The function calls in the main program:
MemoryStream MStream = Otherclass.ImportantStreamManipulator();
StreamReader reader = new StreamReader(MStream);
When I put a breakpoint on the "return MemStream", the "CanRead" property is still set to true. Once I step such that it gets back to my main function, and writes the returned value to MStream, the "CanRead" property is set to false. This then causes an exception in StreamReader saying that MStream could not be read (as the property indicated). The data is in the streams buffer as it should be, but I just can't get it out.
How do I set it so that "CanRead" will report true once it is returned to my main? Or am I misunderstanding how MemoryStream works and how would I accomplish what I want to do?

This is the problem:
using (writer)
{
//Code that writes stuff to the memorystream via streamwriter
return MemStream;
}
You're closing the writer, which closes the MemoryStream. In this case you don't want to do that... although you do need to flush the writer, and rewind the MemoryStream. Just change your code to:
public static MemoryStream ImportantStreamManipulator()
{
// Probably add a comment here stating that the lack of using statements
// is deliberate.
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
// Code that writes stuff to the memorystream via streamwriter
writer.Flush();
stream.Position = 0;
return stream;
}

The StreamWriter takes ownership of the memory stream and when the using statement ends, the MemoryStream is also closed.
See Is there any way to close a StreamWriter without closing its BaseStream?.

As others have stated, the problem is that the Stream is closed when the StreamWriter is closed. One possible way to deal with this is to return a byte array rather than a MemoryStream. This avoids having potentially long running objects that must be disposed by the garbage collector.
public static void Main()
{
OutputData(GetData());
}
public static byte[] GetData()
{
byte[] binaryData = null;
using (MemoryStream ms = new MemoryStream())
using (StreamWriter sw = new StreamWriter(ms))
{
string data = "My test data is really great!";
sw.Write(data);
sw.Flush();
binaryData = ms.ToArray();
}
return binaryData;
}
public static void OutputData(byte[] binaryData)
{
using (MemoryStream ms = new MemoryStream(binaryData))
using (StreamReader sr = new StreamReader(ms))
{
Console.WriteLine(sr.ReadToEnd());
}
}
Another method is to copy the Stream to another stream prior to returning. However, this still has the problem that subsequent access to it with a StreamReader will close that stream.
public static void RunSnippet()
{
OutputData(GetData());
}
public static MemoryStream GetData()
{
MemoryStream outputStream = new MemoryStream();
using (MemoryStream ms = new MemoryStream())
using (StreamWriter sw = new StreamWriter(ms))
{
string data = "My test data is really great!";
sw.Write(data);
sw.Flush();
ms.WriteTo(outputStream);
outputStream.Seek(0, SeekOrigin.Begin);
}
return outputStream;
}
public static void OutputData(MemoryStream inputStream)
{
using (StreamReader sr = new StreamReader(inputStream))
{
Console.WriteLine(sr.ReadToEnd());
}
}

StreamReader ReadToEnd() returns empty string on first attempt

I know this question has been asked before on Stackoverflow, but could not find an explanation.
When I try to read a string from a compressed byte array I get an empty string on the first attempt, on the second I succed and get the string.
Code example:
public static string Decompress(byte[] gzBuffer)
{
if (gzBuffer == null)
return null;
using (var ms = new MemoryStream(gzBuffer))
{
using (var decompress = new GZipStream(ms, CompressionMode.Decompress))
{
using (var sr = new StreamReader(decompress, Encoding.UTF8))
{
string ret = sr.ReadToEnd();
// this is the extra check that is needed !?
if (ret == "")
ret = sr.ReadToEnd();
return ret;
}
}
}
}
All suggestions are appreciated.
- Victor Cassel

I found the bug. It was as Michael suggested in the compression routine. I missed to call Close() on the GZipStream.
public static byte[] Compress(string text)
{
if (string.IsNullOrEmpty(text))
return null;
byte[] raw = Encoding.UTF8.GetBytes(text);
using (var ms = new MemoryStream())
{
using (var compress = new GZipStream (ms, CompressionMode.Compress))
{
compress.Write(raw, 0, raw.Length);
compress.Close();
return ms.ToArray();
}
}
}
What happened was that the data seemed to get saved in a bad state that required two calls to ReadToEnd() in the decompression routine later on to extract the same data. Very odd!

try adding ms.Position = 0 before string ret = sr.ReadToEnd();

Where is gzBuffer coming from? Did you also write the code that is producing the compressed data?
Perhaps the buffer data you have is invalid or somehow incomplete, or perhaps it consists of multiple deflate streams concatenated together.

I hope this helps.
For ByteArray:
static byte[] CompressToByte(string data)
{
MemoryStream outstream = new MemoryStream();
GZipStream compressionStream =
new GZipStream(outstream, CompressionMode.Compress, true);
StreamWriter writer = new StreamWriter(compressionStream);
writer.Write(data);
writer.Close();
return StreamToByte(outstream);
}
static string Decompress(byte[] data)
{
MemoryStream instream = new MemoryStream(data);
GZipStream compressionStream =
new GZipStream(instream, CompressionMode.Decompress);
StreamReader reader = new StreamReader(compressionStream);
string outtext = reader.ReadToEnd();
reader.Close();
return outtext;
}
public static byte[] StreamToByte(Stream stream)
{
stream.Position = 0;
byte[] buffer = new byte[128];
using (MemoryStream ms = new MemoryStream())
{
while (true)
{
int read = stream.Read(buffer, 0, buffer.Length);
if (!(read > 0))
return ms.ToArray();
ms.Write(buffer, 0, read);
}
}
}
You can replace if(!(read > 0)) with if(read <= 0).
For some reason if(read <= 0) isn't displayed corret above.
For Stream:
static Stream CompressToStream(string data)
{
MemoryStream outstream = new MemoryStream();
GZipStream compressionStream =
new GZipStream(outstream, CompressionMode.Compress, true);
StreamWriter writer = new StreamWriter(compressionStream);
writer.Write(data);
writer.Close();
return outstream;
}
static string Decompress(Stream data)
{
data.Position = 0;
GZipStream compressionStream =
new GZipStream(data, CompressionMode.Decompress);
StreamReader reader = new StreamReader(compressionStream);
string outtext = reader.ReadToEnd();
reader.Close();
return outtext;
}

The MSDN Page on the function mentions the following:
If the current method throws an OutOfMemoryException, the reader's position in the underlying Stream object is advanced by the number of characters the method was able to read, but the characters already read into the internal ReadLine buffer are discarded. If you manipulate the position of the underlying stream after reading data into the buffer, the position of the underlying stream might not match the position of the internal buffer. To reset the internal buffer, call the DiscardBufferedData method; however, this method slows performance and should be called only when absolutely necessary.
Perhaps try calling DiscardBufferedData() before your ReadToEnd() and see what it does (I know you aren't getting the exception, but it's all I can think of...)?