Develop Reference

Deserialize JSON from Stream depending on first ~16 characters in .NET Core

I am getting a Gziped JSON which I check for a specific start string and if it's correct I deserialize into my object:
await using var decompressionStream = new GZipStream(externalStream, CompressionMode.Decompress);
var streamReader = new StreamReader(decompressionStream);
string eventString = streamReader.ReadToEnd();
if (!eventString.StartsWith("SomeString"))
{
// Drop it because it can't be deserialized
return;
}
MyObject item = Utf8Json.JsonSerializer.Deserialize<MyObject>(Encoding.UTF8.GetBytes(eventString));
This approach works but it seems to be wasteful to turn the stream into a string and back into an bytes array when Utf8Json can deserialize directly from a stream.
Because 3/4 of incoming streams contain data that is of a different type and can't be deserialized I don't want to put a try/catch block around it because that many exceptions would be too expensive.
Is there a way to peek into the first ~16 chars without consuming them?

How to Trace and Process Stream with Json.Net

I am writing an application which will make a REST HTTP call and receive JSON back in the response body. The response body could be large and the content variable so I am wanting to handle it as a stream and process the content to decide which parts I want to deserialize.
I am able to perform my parsing and deserialize the objects I want using a StreamReader (passing in the HttpResponseMessage.Content, read as a stream) and a JsonTextReader. However, as the JsonTextReader pulls data from the stream, I would also like to trace the raw JSON to a file so that we have the raw response recorded (for diagnostics).
For example, my code:
var serializer = new JsonSerializer()
using (var streamReader = new StreamReader(httpContentStream))
using (var textReader = new JsonTextReader(streamReader))
{
textReader.SupportMultipleContent = true;
while (textReader.Read())
{
// Code which looks at the tokens in the text reader and
// figures out what to throw away, what to deserialize and
// what the next type to deserialize will be (stored in someType)
Type someType = null;
// Deserialize an object that we're interested in.
// This advances the textReader to the next token
// after then end of this object (I.E. More than one token)
var someObject = serializer .Deserialize(textReader, someType);
}
}
Is there a way to also have Json.Net trace out the string that its reading as it pulls characters off the stream? I understand that this could generate a large file!
Thanks in advance for any responses.

NewtonSoft Json Invalid Cast Exception

This is related to my question HTTPClient Buffer Exceeded 2G; Cannot write more bytes to the buffer but is different enough that IMO it warrants a separate question.
In the other question, I'm trying to figure out how to deal with breaking the 2G request buffer. The idea was to use streaming, but I need to deserialize. In talking to Professor Google, I found that I have to use TextReader to stream/deserialize. so my code for that is:
public async Task<API_Json_Special_Feeds.RootObject> walMart_Special_Feed_Lookup(string url)
{
special_Feed_Lookup_Working = true;
HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
using (HttpClient http = new HttpClient(handler))
{
http.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
http.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
url = String.Format(url);
using (var response = await http.GetAsync(url, HttpCompletionOption.ResponseHeadersRead))
{
Console.WriteLine(response);
var serializer = new JsonSerializer();
using (StreamReader sr = new StreamReader(await response.Content.ReadAsStreamAsync()))
{
using (var jsonTextReader = new JsonTextReader(sr))
{
API_Json_Special_Feeds.RootObject root = (API_Json_Special_Feeds.RootObject)serializer.Deserialize(jsonTextReader);
return root;
}
}
}
}
}
Now, as you can see, the return type is strongly typed. The return type of the method matches. Now, I go to the calling line:
API_Json_Special_Feeds.RootObject Items = await net.walMart_Special_Feed_Lookup(specialFeedsURLs[i].Replace("{apiKey}", Properties.Resources.API_Key_Walmart));
So, we have matching types API_Json_Special_Feeds.RootMethod all the way around.
When run, the calling line throws an InvalidCastException:
Undesired Result:
Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'RootObject'
I've checked at the end of the method before return, and the result is indeed cast from an object to API_Json_Special_Feeds.RootMethod before being returned.
Question: somewhere between the return statement and the calling line, the object being returned is being converted from an API_Json_Special_Feeds.RootMethod to a Newtonsoft.Json.Linq.JObject. I can't debug it since there's no code in between. If I cast again in the calling line, I get a "Cannot cast" error. How can I prevent the degradation/changing of this object type?
Many, many thinks for your time, consideration, and for any thoughts or suggestions you can provide!

You need to use the generic overload JsonSerializer.Deserialize<T>()
var root = serializer.Deserialize<API_Json_Special_Feeds.RootObject>(jsonTextReader);
Unlike files generated by BinaryFormatter, JSON files generally do not include c# type information, so it's necessary for the receiving system to specify the expected type.
(There are extensions to the JSON standard in which c# type information can be included in a JSON file -- e.g. Json.NET's TypeNameHandling -- but it is still necessary to deserialize the JSON to an appropriate explicit base class.)
See Deserialize JSON from a file for another example of deserializing a strongly typed c# object from a stream.

How to fix OutOfMemoryException when serializing to JSON string?

I have a Class object which contains a property as byte array in it. This class is a Data Contract to my REST service. The byte array property will take any document whose Max Size is limited to 500MB. When I was trying to consume this service and serializing the object I am getting the Memory Out of exception error. Please find the below image
Below is the code snippet
public static string SerializeJSon<T>(T t)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer ds = new DataContractJsonSerializer(typeof(T));
DataContractJsonSerializerSettings s = new DataContractJsonSerializerSettings();
DateTimeFormat dt = new DateTimeFormat("MM/dd/yyyy");
s.DateTimeFormat = dt;
**ds.WriteObject(stream, t);**
string jsonString = Encoding.UTF8.GetString(stream.ToArray());
stream.Close();
return jsonString;
}

Try "do not use 500mb documents in web service calls". THs is the core problem - you try to use a method call mechanism to transport half a gigabyte of data that likely turns into some gigabyte of in memory objects. This is not what web services are designed to do.

JsonSerializer can't read stream from StreamReader

I can't get the DataContractJsonSerializer object to swallow my stream. When I execute the code with the commented-out line active, I get to see the text provided (and it is a parsable JSON object), so I know that the stream is working fine.
However, for some reason, the compiler complains that the streamReader I'm trying to shove down its throat in ReadObject isn't a Stream. Well, isn't it?!
Argument 1: cannot convert from 'System.IO.StreamReader' to 'System.IO.Stream'
What am I missing and how do I resolve it?
using (StreamReader streamReader = new StreamReader(...))
{
//String responseText = reader.ReadToEnd();
MyThingy thingy = new MyThingy();
DataContractJsonSerializer serializer
= new DataContractJsonSerializer(thingy.GetType());
thingy = serializer.ReadObject(streamReader);
}
I'm adapting this example to work with my stream. Should I approach it from a different angle? If so - how?

You're trying to put in a reader of a stream instead of an actual stream. Skip the using and whatever hides behind the ellipsis (i.e. whatever you put in as an argument when you create an instance of StreamReader), you can probably put that into the ReadObject.
Also, you'll get into problems when reading the data because ReadObject will return an instance of type Object and you'll need to convert it into MyThingy. Since it's a nullable (I'm assuming), you don't have to type cast but rather as-ify it.
MyThingy thingy = new MyThingy();
DataContractJsonSerializer serializer
= new DataContractJsonSerializer(thingy.GetType());
Stream stream = ...;
thingy = serializer.ReadObject(stream) as MyThingy;
You could of course skip the next-to-last line and put the stream directly into the last line.
Courtesy of #JohanLarsson (all Swedes are great, especially those from Stockholm, like me):
In case you can't or don't want to omit the StreamReader declaration in your using statement, I'd suggest that you take a look at BaseStream property to get to it.

You can try this:
using (StreamReader streamReader = new StreamReader(...))
{
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(MyThingy));
MyThingy thingy = (MyThingy) serializer.ReadObject(streamReader.BaseStream);
}

I've been always using this:
// get stuff here
String json = GetJSON();
List<T> result;
using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(json)))
{
var serializer = new DataContractJsonSerializer(typeof(List<T>));
result = (List<T>)serializer.ReadObject(ms);
}