I want to parse the below mentioned JSON and get all values of screenshot.thumbnailUrl. But below are my constraints:
Not all nodes have screenshot. In this example only "weather" and "entities" have it.
I do not know the node names. In this example I didn't know that there would be a node named "weather" or "entities". These nodes are auto generated based on the query i made to get the json.
There are two possible places where screenshot can be present. (1) In the child of rootobject e.g. weather.screenshot (2) In all values of child of rootobject e.g. entities.value[0].screenshot, entities.value[1].screenshot etc.
{ "_type": "SearchResponse", "queryContext": {}, "webPages": {}, "entities": {
"queryScenario": "DominantEntity",
"value": [
{
"_type": "Place",
"id": "https://www.someapi.com/api/v6/#Entities.0",
"screenshot": {
"thumbnailUrl": "http://Screenshot_URL_I_Want",
"width": 285
},
"name": "Seattle",
"entityPresentationInfo": {},
"bingId": "5fbba6b8-85e1-4d41-9444-d9055436e473",
"boundingBox": [],
"weather": {},
"timeZone": "Pacific Standard Time"
}
] }, "images": {}, "weather": {
"id": "https://www.someapi.com/api/v6/#Weather",
"screenshot": {
"thumbnailUrl": "http://Screenshot_URL_I_Want",
"width": 285
},
"location": {},
"currentWeather": {},
"dailyForecast": [] }, "rankingResponse": {} }
This is what worked for me... I am looking for a cleaner solution though...
static async void getJobject(string jsonstring)
{
JObject response = await JObject.Parse(jsonstring);
foreach (var node in response.Children())
{
Console.WriteLine(node.Path);
string propertyPath = node.Path + ".screenshot.thumbnailUrl";
var token = response.SelectToken(propertyPath);
if (token != null)
{
Console.WriteLine("Check this=> " + token.ToString()); //Prints screenshot URL from weather
}
else
{
propertyPath = node.Path + ".value";
token = response.SelectToken(propertyPath);
if (token != null)
{
int count = token.Children().Count();
for (int i = 0; i < count; i++)
{
propertyPath = node.Path + ".value" + "[" + i.ToString() + "]" + ".screenshot.thumbnailUrl";
var mytoken = response.SelectToken(propertyPath);
if (mytoken != null)
{
Console.WriteLine("Check this=> " + mytoken.ToString()); //Prints screenshot URL from entities
}
}
}
}
}
}
You could use code like this to parse the JSon and recursively iterate through it. You will have to probably refine the logic in the lambda of the call to RecursiveDescent to make it correct and robust for your type of JSON:
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApplication2
{
class Program
{
static void Main(string[] args)
{
string json = #"....your JSON ....";
var node = JToken.Parse(json);
RecursiveDescent(node, n =>
{
JToken url = n["thumbnailUrl"];
if (url != null && url.Type == JTokenType.String)
{
var nodeWeWant = url?.Parent?.Parent?.Parent?.Parent;
Console.WriteLine(nodeWeWant.ToString());
}
});
}
static void RecursiveDescent(JToken node, Action<JObject> action)
{
if (node.Type == JTokenType.Object)
{
action((JObject)node);
foreach (JProperty child in node.Children<JProperty>())
RecursiveDescent(child.Value, action);
}
else if (node.Type == JTokenType.Array)
{
foreach (JToken child in node.Children())
RecursiveDescent(child, action);
}
}
}
}
Related
I have data in provinces collection like this:
{
"_id": {
"$oid": "63dc7ff82e7e5e91c0f1cd87"
},
"province": "province1",
"districts": [
{
"district": "district1",
"sub_districts": [
{
"sub_district": "sub_district1",
"zip_codes": [
"zip_code1"
]
},
{
"sub_district": "sub_district2",
"zip_codes": [
"zip_code2"
]
},
],
},
],
}
This is how I get a list of sub_district for now:
- I search for province using Builders.Filter.
- Use foreach to get districts array (In province collection) and use if-statement to check if district equal searchDistrict.
- Get sub_districts array in that distric.
Source code:
public static List<string> MongoDbSelectSubDistrict(string searchProvince, string searchDistrict)
{
List<string> subDistrictList = new List<string>();
try
{
var provincesCollection = _db.GetCollection<BsonDocument>("provinces");
var builder = Builders<BsonDocument>.Filter;
var filter = builder.Empty;
if (searchProvince != "")
{
var provinceFilter = Builders<BsonDocument>.Filter.Eq("province", searchProvince);
filter &= provinceFilter;
}
/*
//***Need to be revised***
if (searchDistrict != "")
{
var districtFilter = Builders<BsonDocument>.Filter.Eq("provinces.district", searchDistrict);
filter &= districtFilter;
}
*/
var queryProvinces = provincesCollection.Find(filter).ToList();
foreach (BsonDocument queryProvince in queryProvinces)
{
BsonArray districtArray = queryProvince.GetValue("districts").AsBsonArray;
foreach (BsonDocument districtDocument in districtArray)
{
string district = districtDocument.GetValue("district").ToString();
if (district == searchDistrict) //***Need to be revised***
{
BsonArray subDistrictArray = districtDocument.GetValue("sub_districts").AsBsonArray;
foreach (BsonDocument subDistrictDocument in subDistrictArray)
{
string subDistrict = subDistrictDocument.GetValue("sub_district").ToString();
subDistrictList.Add(subDistrict);
}
}
}
}
}
catch (TimeoutException ex)
{
}
return subDistrictList;
}
Is there any efficient way to get this?
This is what I want:
[
{
"sub_district": "sub_district1",
"zip_codes": [
"zip_code1"
]
},
{
"sub_district": "sub_district2",
"zip_codes": [
"zip_code2"
]
},
]
And one more question: if I want to search for sub_district in the collection, how do I get this without looping in sub_districts array?
I'm trying to display the simple element of the array or value, below is my JSON string(an API from python pandas)
{
"msg": "success",
"state": 10000,
"data": {
"data": [
{
"index": 0,
"test_1": 110,
"test_2": "000001",
"test_3": "CN",
"test_4": "Bank",
"test_5": 893,
"test_6": 229
}
],
"schema": {
"fields": [
{
"type": "integer",
"name": "index"
},
{
"type": "string",
"name": "test_1"
},
{
"type": "string",
"name": "test_2"
},
{
"type": "number",
"name": "test_3"
},
{
"type": "number",
"name": "test_4"
},
{
"type": "number",
"name": "test_5"
},
{
"type": "string",
"name": "test_6"
}
],
"pandas_version": "0.20.0",
"primaryKey": [
"index"
]
}
}
}
Below code of C# is the query that I'm using, TestDataset.cs:
using System;
using System.IO;
using System.Net;
using Newtonsoft.Json.Linq;
using ExcelDna.Integration;
using Excel = Microsoft.Office.Interop.Excel;
namespace Test_Atune
{
public class Request
{
Excel.Application ExcelApp = (Excel.Application)ExcelDnaUtil.Application;
public object GetDatasetFromUrl(string root, string path, string headval, Excel.Range excelAddress)
{
string historicalData= "{}";
var webConnection = new WebClient();
webConnection.Headers.Add("Host", headval);
try
{
historicalData = webConnection.DownloadString(root+path);
}
catch (WebException ex)
{
string error_str = ex.Message;
if (ex.Status.ToString() == "ConnectFailure" || ex.Status.ToString() == "Timeout")
{
root = Constant.setURL_ROOT_COMMON(root);
try
{
historicalData= webConnection.DownloadString(root + path);
}
catch (WebException ex2)
{
return "";
}
}
}
finally
{
webConnection.Dispose();
}
JObject jsonFeed = JObject.Parse(historicalData);
Excel.Range range = excelAddress;
JObject B = new JObject();
JArray data = (JArray)jsonFeed["data"]["data"];
JArray columns = (JArray)jsonFeed["data"]["schema"]["fields"];
int rowNum = data.Count;
int colNum = columns.Count;
Excel.Range range_head = ExcelApp.Cells[range.Row + 1, range.Column];
range_head = range_head.get_Resize(1, colNum);
Excel.Range range_data = ExcelApp.Cells[range.Row + 2, range.Column];
range_data = range_data.get_Resize(rowNum, colNum);
// write header
object[,] headerData = new object[1, colNum];
for (int iCol = 0; iCol < colNum; iCol++)
{
headerData[0, iCol] = columns[iCol]["name"];
}
range_head.Value2 = headerData;
// write data
object[,] cellData = new object[rowNum, colNum];
int iRow = 0;
foreach (JObject jo in data)
{
var a = jo["test_code"];
for (int iCol = 0; iCol < colNum; iCol++)
{
if (columns[iCol]["test_1"].ToString() == "string")
{
cellData[iRow, iCol] = "'" + jo[columns[iCol]["name"].ToString()];
}
else
{
cellData[iRow, iCol] = jo[columns[iCol]["name"].ToString()];
}
}
iRow += 1;
}
range_data.Value2 = cellData;
return "Total" + rowNum.ToString() + "cells";
}
}
}
And below is the code of request.cs
using ExcelDna.Integration;
using Excel = Microsoft.Office.Interop.Excel;
namespace Test_Atune
{
public class Dataset
{
public static string baseurl = Constant.URL_ROOT_COMMON;
public static string headval = Constant.HEADVAL_COMMON;
public static Request request = new Request();
[ExcelFunction(Category = "test", IsMacroType = true, Description = "test dataset")]
public static object TEST_DATASET(
[ExcelArgument(Description = "test_code")] string test_1,
[ExcelArgument(Description = "YYYYMMDD")] string test_2,
[ExcelArgument(Description = "YYYYMMDD")] string test_3
)
{
string parstr = #"/test_dataset/?" +
#"test_1=" + test_1 +
#"&test_2=" + test_2 +
#"&test_3=" + test_3;
ExcelReference caller = (ExcelReference)XlCall.Excel(XlCall.xlfCaller);
Excel.Range rg = caller.ToPiaRange();
return ExcelAsyncUtil.Run("TEST_DATASET",
new object[] { parstr },
() => request.GetDatasetFromUrl(Constant.URL_ROOT_COMMON, parstr, headval, rg));
}
}
}
And I got the following error on JArray columns = (JArray)jsonFeed["data"]["schema"]["fields"];
System.NullReferenceException HResult=0x80004003 message=Object reference not set to an instance of an object.
I tried to debug and I got below results,
Name: historicalData, Value:"{}"; Name: jsonFeed,Vaule:Null; Name:B,Vaule:{{}}, Name:data, Value:"null"
I am very new to C#, is that an array,domain or url problem, or anything else? How can I do that? thank you so much for any advice.
Error: Object reference not set to an instance of an object. This means that the one or all of the data under ["data"] or ["data"]["schema"] or ["data"]["schema"]["fields"] does not exist.
I just tested your code and the following code works just fine based on the given json.
JArray columns = (JArray)jsonFeed["data"]["schema"]["fields"];
Suggestion: Use the debugger and see if the value of your jsonFeed is correct because thats the only thing that I cannot reproduce. If you are not able to use the debugger, log the parsed object by using .ToString() on the parsed object. (jsonFeed.ToString())
NOTE: There is an issue with your json. "test_3": CN, is suposed to be "test_3": "CN", with quotes on CN.
I have a JSON string, which is srtuctered as per the below.
"total": 5,
"filtered": 5,
"items": [
{
"assignedProducts": [
"antivirus"
],
"cloned": false,
"device_encryption_status_unmanaged": false,
"java_id": "2408cf5b-669c-434e-ac4c-a08d93c40e6a",
"last_activity": "2019-09-20T12:36:22.175Z",
"last_user_id": "5c6bc52305f05316ba18db06",
"heartbeat_utm_name": null,
"group_full_name": null,
"is_adsync_group": false,
"is_caching_proxy": false,
"info": {
"processor_architecture": "x64",
"osMajorVersion": 10,
"computer_name": "INDPC01",
"isServer": false,
"osInstallationType": "Client",
"fqdn": "INDPC01",
"osName": "Windows 10 Pro N",
"id": "4280fcb5-66c9-34e4-cac4-0ad8394ce0a6",
"name": "INDPC01"
},
I am using the following code to get values from a JSON string.
var resultObjects = AllChildren(JObject.Parse(response.Content))
.First(c => c.Type == JTokenType.Array && c.Path.Contains("items"))
.Children<JObject>();
foreach (JObject result in resultObjects)
{
foreach (JProperty property in result.Properties())
{
ListBox1.Items.Add(property.Name.ToString() + " - " + property.Value.ToString());
}
}
}
private static IEnumerable<JToken> AllChildren(JToken json)
{
foreach (var c in json.Children())
{
yield return c;
foreach (var cc in AllChildren(c))
{
yield return cc;
}
}
}
I am able to get all the values for everything within "Items", however I am not sure how to access the child JSON objects, for example the information under "info"
You can access childs using indexer, e.g.:
// ...
foreach (JObject result in resultObjects)
{
var architecture = result["info"]["processor_architecture"].ToString();
// ...
}
Note that it might be an easier approach to convert the JSON to an object, e.g. using JsonConvert, instead using JObject.Parse.
I want to scrape chrome bookmarks using json object. What I want to do is get all bookmarks from 1 folder. That is the structure of the json:
{
"checksum": "d499848083c2c2e3a38f547da4cbad7c",
"roots": {
"bookmark_bar": {
"children": [ {
"children": [ {
"url": "https://www.example.com/"
}, {
"url": "https://www.another-example.com/"
} ],
"name": "foo",
} ],
"name": "Menu bookmarks",
},
"other": {
"name": "Another bookmarks",
},
"synced": {
"name": "Phone bookmarks",
}
},
"version": 1
}
In this case I want to get urls from folder foo. I am using Json.NET to convert string into object. This code:
string input = File.ReadAllText(bookmarksLocation);
using (StringReader reader = new StringReader(input))
using (JsonReader jsonReader = new JsonTextReader(reader)) {
JsonSerializer serializer = new JsonSerializer();
var o = (JToken)serializer.Deserialize(jsonReader);
var allChildrens = o["roots"]["bookmark_bar"]["children"];
var fooFolder = allChildrens.Where(x => x["name"].ToString() == "foo");
foreach (var item in fooFolder["children"]) {
Console.WriteLine(item["url"].ToString());
}
Console.ReadKey();
}
Is giving me an error:
Cannot apply indexing with [] to an expression of type 'IEnumerable<JToken>'
Can you tell me what I did wrong?
there is 1 loop is missing:
var fooFolder = allChildrens.Where(x => x["name"].ToString() == "foo");
foreach (var folder in fooFolder)
{
foreach (var item in folder["children"])
{
Console.WriteLine(item["url"].ToString());
}
}
I am fetching some data from external Webservice and parsing it to json using Newtonsoft.Json.Linq
like this
JObject o = JObject.Parse(json);
JArray sizes = (JArray) o["data"];
Now the Sizes looks like this
{
[
{
"post_id": "13334556777742_6456",
"message": "messagecomes her",
"attachment": {
"media": [
{
"href": "http://onurl.html",
"alt": "",
"type": "link",
"src": "http://myurl.jpg"
}
],
"name": "come to my name",
"href": "http://mydeeplink.html",
"description": "",
"properties": [],
},
}
]
}
I need to get "src": "http://myurl.jpg"element from this Json array.
I have tried:
foreach (JObject obj in sizes)
{
JArray media = (JArray)obj["attachment"];
foreach (JObject obj1 in media)
{
var src = obj1["src"];
}
}
But it's throwing an error:
Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'Newtonsoft.Json.Linq.JArray'.
at this line
JArray media = (JArray)obj["attachment"];
Can any one give me a hand on this?
Try fix line
JArray media = (JArray)(obj["attachment"]);
to
JArray media = (JArray)(obj["attachment"]["media"]);
This is how I handled a scenario that sounds just like yours:
public static IList<Entity> DeserializeJson(JToken inputObject)
{
IList<Entity> deserializedObject = new List<Entity>();
foreach (JToken iListValue in (JArray)inputObject["root"])
{
Entity entity = new Entity();
entity.DeserializeJson(iListValue);
deserializedObject.Add(entity);
}
return deserializedObject;
}
public virtual void DeserializeJson(JToken inputObject)
{
if (inputObject == null || inputObject.Type == JTokenType.Null)
{
return;
}
inputObject = inputObject["entity"];
JToken assertions = inputObject["assertions"];
if (assertionsValue != null && assertionsValue.Type != JTokenType.Null)
{
Assertions assertions = new Assertions();
assertions.DeserializeJson(assertionsValue);
this.Assertions = assertions;
}
// Continue Parsing
}