Develop Reference

List<JToken> returned as empty array in JSON

I am trying to read a json file and parse it using
jArray = JArray.Parse(json);
And filtering that array based on some conditions and returning same in ActionREsult OK
var categories = jArray.SelectTokens("[*]")
.Where(token => (string)token.SelectToken(jPath) == displayLevel);
return Ok(categories);
But on client side I am receiving it as json string with empty array like this
[[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]],[[[]],[[]],[[]],[[]],[[]]]]
All this was working before.

Convert a list of strings to square brackets to json

I'm trying to update a JSON file with converted list of strings to JSON square brackets "list".
My strings list:
var animals = new List<string>() { "bird", "dog" };
Using this code:
string json = File.ReadAllText(filePath);
dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(json);
var serializedObject = JsonConvert.SerializeObject(animals);
jsonObj["animals"] = serializedObject;
string output = Newtonsoft.Json.JsonConvert.SerializeObject(jsonObj, Newtonsoft.Json.Formatting.Indented);
File.WriteAllText(filePath, output);
Old JSON file:
{
"animals": ["cat", "fox"]
}
The new JSON file should be:
{
"animals": ["bird", "dog"]
}
But what i get is:
{
"animals": "[\"bird\", \"dog\"]"
}
Any help is appreciated!
Thanks

Your serializedObject is a string, but you don't need it at all it.
As you don't deserialize to a concrete type, your jsonObj["animals"] is just a JArray. So you need this:
dynamic jsonObj = JsonConvert.DeserializeObject(json);
jsonObj["animals"] = JArray.FromObject(animals);
Now you can serialize it back through JsonConvert.SerializeObject.

If that jsonObj was a regular object you could just set the value of the animals property. The same would work if it was an ExpandoObject. JsonConvert.DeserializeObject(json) though generates a Json.Net type whose data must be valid Json.NET types.
You can assing the list contents as a JArray, eg :
var animals = new List<string>() { "bird", "dog" };
dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject("{'moo':1}");
jsonObj.animals= new JArray(animals);
var result=JsonConvert.SerializeObject(jsonObj);
The result will be :
{"moo":1,"animals":["bird","dog"]}
Adding a new property will work only if the file contains a JSON dictionary. If you know that the file will alway contain a dictionary, you can cast the deserialization result to JObject, and add the new property through JObject's indexer :
var jsonObj = (JObject)JsonConvert.DeserializeObject("{'moo':1}");
jsonObj["animals"]= new JArray(animals);

How to parse huge JSON file as stream in Json.NET?

I have a very, very large JSON file (1000+ MB) of identical JSON objects. For example:
[
{
"id": 1,
"value": "hello",
"another_value": "world",
"value_obj": {
"name": "obj1"
},
"value_list": [
1,
2,
3
]
},
{
"id": 2,
"value": "foo",
"another_value": "bar",
"value_obj": {
"name": "obj2"
},
"value_list": [
4,
5,
6
]
},
{
"id": 3,
"value": "a",
"another_value": "b",
"value_obj": {
"name": "obj3"
},
"value_list": [
7,
8,
9
]
},
...
]
Every single item in the root JSON list follows the same structure and thus would be individually deserializable. I already have the C# classes written to receive this data, and deserializing a JSON file containing a single object without the list works as expected.
At first, I tried to just directly deserialize my objects in a loop:
JsonSerializer serializer = new JsonSerializer();
MyObject o;
using (FileStream s = File.Open("bigfile.json", FileMode.Open))
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
while (!sr.EndOfStream)
{
o = serializer.Deserialize<MyObject>(reader);
}
}
This didn't work, threw an exception clearly stating that an object is expected, not a list. My understanding is that this command would just read a single object contained at the root level of the JSON file, but since we have a list of objects, this is an invalid request.
My next idea was to deserialize as a C# List of objects:
JsonSerializer serializer = new JsonSerializer();
List<MyObject> o;
using (FileStream s = File.Open("bigfile.json", FileMode.Open))
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
while (!sr.EndOfStream)
{
o = serializer.Deserialize<List<MyObject>>(reader);
}
}
This does succeed. However, it only somewhat reduces the issue of high RAM usage. In this case it does look like the application is deserializing items one at a time, and so is not reading the entire JSON file into RAM, but we still end up with a lot of RAM usage because the C# List object now contains all of the data from the JSON file in RAM. This has only displaced the problem.
I then decided to simply try taking a single character off the beginning of the stream (to eliminate the [) by doing sr.Read() before going into the loop. The first object then does read successfully, but subsequent ones do not, with an exception of "unexpected token". My guess is this is the comma and space between the objects throwing the reader off.
Simply removing square brackets won't work since the objects do contain a primitive list of their own, as you can see in the sample. Even trying to use }, as a separator won't work since, as you can see, there are sub-objects within the objects.
What my goal is, is to be able to read the objects from the stream one at a time. Read an object, do something with it, then discard it from RAM, and read the next object, and so on. This would eliminate the need to load either the entire JSON string or the entire contents of the data into RAM as C# objects.
What am I missing?

This should resolve your problem. Basically it works just like your initial code except it's only deserializing object when the reader hits the { character in the stream and otherwise it's just skipping to the next one until it finds another start object token.
JsonSerializer serializer = new JsonSerializer();
MyObject o;
using (FileStream s = File.Open("bigfile.json", FileMode.Open))
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
while (reader.Read())
{
// deserialize only when there's "{" character in the stream
if (reader.TokenType == JsonToken.StartObject)
{
o = serializer.Deserialize<MyObject>(reader);
}
}
}

I think we can do better than the accepted answer, using more features of JsonReader to make a more generalized solution.
As a JsonReader consumes tokens from a JSON, the path is recorded in the JsonReader.Path property.
We can use this to precisely select deeply nested data from a JSON file, using regex to ensure that we're on the right path.
So, using the following extension method:
public static class JsonReaderExtensions
{
public static IEnumerable<T> SelectTokensWithRegex<T>(
this JsonReader jsonReader, Regex regex)
{
JsonSerializer serializer = new JsonSerializer();
while (jsonReader.Read())
{
if (regex.IsMatch(jsonReader.Path)
&& jsonReader.TokenType != JsonToken.PropertyName)
{
yield return serializer.Deserialize<T>(jsonReader);
}
}
}
}
The data you are concerned with lies on paths:
[0]
[1]
[2]
... etc
We can construct the following regex to precisely match this path:
var regex = new Regex(#"^\[\d+\]$");
it now becomes possible to stream objects out of your data (without fully loading or parsing the entire JSON) as follows
IEnumerable<MyObject> objects = jsonReader.SelectTokensWithRegex<MyObject>(regex);
Or if we want to dig even deeper into the structure, we can be even more precise with our regex
var regex = new Regex(#"^\[\d+\]\.value$");
IEnumerable<string> objects = jsonReader.SelectTokensWithRegex<string>(regex);
to only extract value properties from the items in the array.
I've found this technique extremely useful for extracting specific data from huge (100 GiB) JSON dumps, directly from HTTP using a network stream (with low memory requirements and no intermediate storage required).

.NET 6
This is easily done with the System.Text.Json.JsonSerializer in .NET 6:
using (FileStream? fileStream = new FileStream("hugefile.json", FileMode.Open))
{
IAsyncEnumerable<Person?> people = JsonSerializer.DeserializeAsyncEnumerable<Person?>(fileStream);
await foreach (Person? person in people)
{
Console.WriteLine($"Hello, my name is {person.Name}!");
}
}

Here is another easy way to parse large JSON file using Cinchoo ETL, an open source library (Uses JSON.NET under the hood to parse the json in stream manner)
using (var r = ChoJSONReader<MyObject>.LoadText(json)
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
Sample fiddle: https://dotnetfiddle.net/i5qJ5R

Is this what you're looking for? Found on a previous question
The current version of Json.net does not allow you to use the accepted answer code. A current alternative is:
public static object DeserializeFromStream(Stream stream)
{
var serializer = new JsonSerializer();
using (var sr = new StreamReader(stream))
using (var jsonTextReader = new JsonTextReader(sr))
{
return serializer.Deserialize(jsonTextReader);
}
}
Documentation: Deserialize JSON from a file stream

loop on data returned after deserializing

I get data from instagram api. So i use below
var responseStream = webRequest.GetResponse().GetResponseStream();
Encoding encode = System.Text.Encoding.Default;
using (StreamReader reader = new StreamReader(responseStream, encode))
{
var serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
var jsonObject = serializer.DeserializeObject(reader.ReadToEnd());
}
I dont know how to loop on this data. I tried
foreach (var item in jsonObject)
{
}
But it gives compile time error Type object is not enumerable
How do i loop?

Your variable jsonObject is an object which is not an enumerable.
You must specify an enumerable type when deserializing :
var jsonObject = serialize.Deserialize<enumerableType>(reader.ReadToEnd());

Inserting a Row in BigQuery with Nested Record using C# API

I'm trying to insert a row (with nested records) into BigQuery using the C# API. I'm able to insert a row (w/ nested records) using the JavaScript API. But using the C# API i'm getting error saying: "Repeated field must be imported as a JSON Array". He is a simple row that I was able to insert using the JavaScript API
var json = {'rows':[{'json':
{"inputs" : [{
"Age":"10"
}]}}]};
This works fine in JS, but I'm unclear how to do this in C#.
Here is my attempt:
var r = new TableDataInsertAllRequest.RowsData();
r.Json = new Dictionary<string, object>();
var dict = new Dictionary<string, object>();
dict.Add("Age", "10");
r.Json.Add("inputs", dict);
Also I tried using JSON API
string json = JsonConvert.SerializeObject(input, jsonSettings);
var dict = JsonConvert.DeserializeObject<Dictionary<string, object>>(json, jsonSettings);
r.Json.Add("jsonInputs", dict);
Here is the API Doc for RowsData https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/csharp/latest/classGoogle_1_1Apis_1_1Bigquery_1_1v2_1_1Data_1_1TableDataInsertAllRequest_1_1RowsData.html
Its somewhat vague or unclear how to do nested records. I tried just writing straight JSON but getting the same errors.
Any help would be much appreciated.
Thanks,

From the json example that works there are two arrays. I don't see any arrays in either of two failing examples you provide.
I suspect adding the arrays will fix your problem. Consider parsing the known-working json string into a json object and using that as a quick test.

The following code snippet successfully works for bigquery:
JArray inputs = new JArray();
JObject inputOne = new JObject(new JProperty("Age", "12"));
inputOne.Add(new JProperty("BirthDate", "1234"));
r.Json.Add("inputs", inputs);