This is related to my question HTTPClient Buffer Exceeded 2G; Cannot write more bytes to the buffer but is different enough that IMO it warrants a separate question.
In the other question, I'm trying to figure out how to deal with breaking the 2G request buffer. The idea was to use streaming, but I need to deserialize. In talking to Professor Google, I found that I have to use TextReader to stream/deserialize. so my code for that is:
public async Task<API_Json_Special_Feeds.RootObject> walMart_Special_Feed_Lookup(string url)
{
special_Feed_Lookup_Working = true;
HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
using (HttpClient http = new HttpClient(handler))
{
http.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
http.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
url = String.Format(url);
using (var response = await http.GetAsync(url, HttpCompletionOption.ResponseHeadersRead))
{
Console.WriteLine(response);
var serializer = new JsonSerializer();
using (StreamReader sr = new StreamReader(await response.Content.ReadAsStreamAsync()))
{
using (var jsonTextReader = new JsonTextReader(sr))
{
API_Json_Special_Feeds.RootObject root = (API_Json_Special_Feeds.RootObject)serializer.Deserialize(jsonTextReader);
return root;
}
}
}
}
}
Now, as you can see, the return type is strongly typed. The return type of the method matches. Now, I go to the calling line:
API_Json_Special_Feeds.RootObject Items = await net.walMart_Special_Feed_Lookup(specialFeedsURLs[i].Replace("{apiKey}", Properties.Resources.API_Key_Walmart));
So, we have matching types API_Json_Special_Feeds.RootMethod all the way around.
When run, the calling line throws an InvalidCastException:
Undesired Result:
Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'RootObject'
I've checked at the end of the method before return, and the result is indeed cast from an object to API_Json_Special_Feeds.RootMethod before being returned.
Question: somewhere between the return statement and the calling line, the object being returned is being converted from an API_Json_Special_Feeds.RootMethod to a Newtonsoft.Json.Linq.JObject. I can't debug it since there's no code in between. If I cast again in the calling line, I get a "Cannot cast" error. How can I prevent the degradation/changing of this object type?
Many, many thinks for your time, consideration, and for any thoughts or suggestions you can provide!
You need to use the generic overload JsonSerializer.Deserialize<T>()
var root = serializer.Deserialize<API_Json_Special_Feeds.RootObject>(jsonTextReader);
Unlike files generated by BinaryFormatter, JSON files generally do not include c# type information, so it's necessary for the receiving system to specify the expected type.
(There are extensions to the JSON standard in which c# type information can be included in a JSON file -- e.g. Json.NET's TypeNameHandling -- but it is still necessary to deserialize the JSON to an appropriate explicit base class.)
See Deserialize JSON from a file for another example of deserializing a strongly typed c# object from a stream.
Related
I am getting a Gziped JSON which I check for a specific start string and if it's correct I deserialize into my object:
await using var decompressionStream = new GZipStream(externalStream, CompressionMode.Decompress);
var streamReader = new StreamReader(decompressionStream);
string eventString = streamReader.ReadToEnd();
if (!eventString.StartsWith("SomeString"))
{
// Drop it because it can't be deserialized
return;
}
MyObject item = Utf8Json.JsonSerializer.Deserialize<MyObject>(Encoding.UTF8.GetBytes(eventString));
This approach works but it seems to be wasteful to turn the stream into a string and back into an bytes array when Utf8Json can deserialize directly from a stream.
Because 3/4 of incoming streams contain data that is of a different type and can't be deserialized I don't want to put a try/catch block around it because that many exceptions would be too expensive.
Is there a way to peek into the first ~16 chars without consuming them?
I am writing an application which will make a REST HTTP call and receive JSON back in the response body. The response body could be large and the content variable so I am wanting to handle it as a stream and process the content to decide which parts I want to deserialize.
I am able to perform my parsing and deserialize the objects I want using a StreamReader (passing in the HttpResponseMessage.Content, read as a stream) and a JsonTextReader. However, as the JsonTextReader pulls data from the stream, I would also like to trace the raw JSON to a file so that we have the raw response recorded (for diagnostics).
For example, my code:
var serializer = new JsonSerializer()
using (var streamReader = new StreamReader(httpContentStream))
using (var textReader = new JsonTextReader(streamReader))
{
textReader.SupportMultipleContent = true;
while (textReader.Read())
{
// Code which looks at the tokens in the text reader and
// figures out what to throw away, what to deserialize and
// what the next type to deserialize will be (stored in someType)
Type someType = null;
// Deserialize an object that we're interested in.
// This advances the textReader to the next token
// after then end of this object (I.E. More than one token)
var someObject = serializer .Deserialize(textReader, someType);
}
}
Is there a way to also have Json.Net trace out the string that its reading as it pulls characters off the stream? I understand that this could generate a large file!
Thanks in advance for any responses.
I'm accessing the tomtom json api, and the api either returns me an array of objects, or a single object, when an error has happen.
Example:
[{"driverno": "...
Error Example:
{"errorCode": "8011","errorMsg": "request quota reached, error code: 8011"}
The data is accessed WebRequest, WebResponse and they return a stream, which can then be passed to a DataContractJsonSerializer. However, I can't create a serialization class, which accepts both forms of JSON, and the stream can't be passed twice, because the seek function is not supported.
Is there a way, to create a serialization class which supports both types of JSON input?
I found a workaround, where I copy the Stream to a MemoryStream, which enables seeking. I'm not completly settisfied with th solution, becuase it does a Stream copying and the DataContractJsonSerializer twice.
Sample:
string text = File.ReadAllText(PAHT);
text = Regex.Replace(text, "\\{[\\n\\r ]*\"__type", "{\"__type");
// copy to MemoryStream
using (MemoryStream dataStream = new MemoryStream(Encoding.UTF8.GetBytes(text)))
{
DataContractJsonSerializer errorDeserializer = new DataContractJsonSerializer(typeof(RequestError));
RequestError errorSerilaized = (RequestError)errorDeserializer.ReadObject(dataStream);
// check if an error happened
if (errorSerilaized.errorCode == null)
{
// seek the stream to position 0
dataStream.Position = 0;
DataContractJsonSerializer _deserializer = new DataContractJsonSerializer(typeof(NoneErrorSerializationClass));
NoneErrorSerializationClass tripReportsSerialized = (NoneErrorSerializationClass)_deserializer.ReadObject(dataStream);
// ...
}
else
{
MessageBox.Show(errorSerilaized.errorMsg);
}
}
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
If it helps, I'm using the CampBX API to get funds in my account. I wrote the following code to make the API Call:
using (var wb = new WebClient())
{
String url = "https://CampBX.com/api/myfunds.php";
var data = new NameValueCollection();
data["user"] = "USERNAME";
data["pass"] = "PASSWORD";
var response = wb.UploadValues(url, "POST", data);
}
WebClient.UploadValues() returns a byte[] and I have no idea how to correctly parse that.
Here is the CampBX info, under Account Balances.
Simply put, you need to use a JSON parser. Personally I like Newtonsoft.Json and it is what I will use in this example.
The first step is converting the byte[] to a sequence of characters, either a string object or a TextReader. The second step is to pass this information to the parser. So, in your case, the code would look something like this:
JToken parsedToken;
using (var responseReader = new StreamReader(new MemoryStream(response))) {
parsedToken = JToken.ReadFrom(responseReader);
}
The parsedToken object can then be used to extract whatever data you need. (See the documentation for information on extracting data from a JToken object.)
Note that WebClient.UploadValues() discards the information regarding the response entity's character encoding. StreamReader will use UTF-8 encoding by default, which is sufficient to parse UTF-8 or ASCII. Depending on the JSON encoder used by the server, the response may always be ASCII-compatible, so you might not have to worry about it. Nevertheless, it's something you should investigate.
DataContractJsonSerializer built-in object will be your friend here, provided you know what is the internal structure of the returned object (or can at least guess it, from the JSON).
The steps are:
Define a contract class to hold the deserialized JSON object
namespace AppNameSpace
{
[DataContract] /* Place this inside your app namespace */
internal class iResponse /*Name this class appropriately */
{
[DataMember]
internal string field1;
[DataMember]
internal string field2;
[DataMember]
internal Int32 field3;
}
...
}
The actual parsing itself is about three lines
DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(iOpenParams));
MemoryStream stream1 = new MemoryStream(response);
iResponse resp_json = (iResponse)ser.ReadObject(stream1);
For more details and examples, refer to: http://msdn.microsoft.com/en-us/library/bb412179.aspx
My solution is simpler:
Object retorno;
var response = wb.UploadValues(url, "POST", data);
using (var responseReader = new StreamReader(new MemoryStream(response)))
{
retorno = JsonConvert.DeserializeObject<Object>(responseReader.ReadToEnd());
}
I can't get the DataContractJsonSerializer object to swallow my stream. When I execute the code with the commented-out line active, I get to see the text provided (and it is a parsable JSON object), so I know that the stream is working fine.
However, for some reason, the compiler complains that the streamReader I'm trying to shove down its throat in ReadObject isn't a Stream. Well, isn't it?!
Argument 1: cannot convert from 'System.IO.StreamReader' to 'System.IO.Stream'
What am I missing and how do I resolve it?
using (StreamReader streamReader = new StreamReader(...))
{
//String responseText = reader.ReadToEnd();
MyThingy thingy = new MyThingy();
DataContractJsonSerializer serializer
= new DataContractJsonSerializer(thingy.GetType());
thingy = serializer.ReadObject(streamReader);
}
I'm adapting this example to work with my stream. Should I approach it from a different angle? If so - how?
You're trying to put in a reader of a stream instead of an actual stream. Skip the using and whatever hides behind the ellipsis (i.e. whatever you put in as an argument when you create an instance of StreamReader), you can probably put that into the ReadObject.
Also, you'll get into problems when reading the data because ReadObject will return an instance of type Object and you'll need to convert it into MyThingy. Since it's a nullable (I'm assuming), you don't have to type cast but rather as-ify it.
MyThingy thingy = new MyThingy();
DataContractJsonSerializer serializer
= new DataContractJsonSerializer(thingy.GetType());
Stream stream = ...;
thingy = serializer.ReadObject(stream) as MyThingy;
You could of course skip the next-to-last line and put the stream directly into the last line.
Courtesy of #JohanLarsson (all Swedes are great, especially those from Stockholm, like me):
In case you can't or don't want to omit the StreamReader declaration in your using statement, I'd suggest that you take a look at BaseStream property to get to it.
You can try this:
using (StreamReader streamReader = new StreamReader(...))
{
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(MyThingy));
MyThingy thingy = (MyThingy) serializer.ReadObject(streamReader.BaseStream);
}
I've been always using this:
// get stuff here
String json = GetJSON();
List<T> result;
using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(json)))
{
var serializer = new DataContractJsonSerializer(typeof(List<T>));
result = (List<T>)serializer.ReadObject(ms);
}