Lets see, the json can be dynamic and can probably have number of nested arrays within any property.
Example:
{
"items": [
{
"id": "0001",
"name": "Cake",
"batters": {
"batter": [
{
"id": "1001",
"type": "Regular"
},
{
"id": "1002",
"type": "Chocolate"
},
{
"dry": [
{
"id": "1003",
"type": "Devil's Food"
}
]
}
],
"other": [
{
"id": "1004",
"type": "Home Food"
}
]
},
"topping": [
{
"id": "5002",
"type": "Glazed"
},
{
"id": "5005",
"type": "Sugar"
}
]
},
{
"id": "0002",
"name": "Sweets"
}
]
}
A simple list should return elements as:
[
{
"id": "1001",
"type": "Regular"
},
{
"id": "1002",
"type": "Chocolate"
},
{
"id": "1003",
"type": "Devil's Food"
},
{
"id": "1004",
"type": "Home Food"
},
{
"id": "5002",
"type": "Glazed"
},
{
"id": "5005",
"type": "Sugar"
},
{
"id": "0002",
"name": "Sweets"
}
]
Please note:
Json can by anything, no property can be used for extraction , just knowing that what needed is stuff inside an JArray.
What i have tried so far but its just a start:
public static bool ParseJsonArray(JToken token, List<string> extracts, string parentLocation = "")
{
if (token.HasValues)
{
foreach (JToken child in token.Children())
{
if (token.Type == JTokenType.Array)
{
parentLocation += ((JProperty)token).Name;
extracts.Add(token.ToString());
}
ParseJsonArray(child, extracts, parentLocation);
}
return true;
}
else
{
return false;
}
}
token here is the parsed dynamic json.
It appears as though you want to recursively find all JArray entries that do not themselves contain nested arrays. Let's call these "leaf" array entries. I say that because you don't include the following non-leaf entry in your results:
{
"id": "0001",
"name": "Cake"
}
That being said, you can find leaf array entries with the following extension method:
public static class JsonExtensions
{
public static IEnumerable<JToken> LeafArrayEntries(this JContainer container)
{
var nonLeafEntries = new HashSet<JToken>(container.DescendantsAndSelf()
.OfType<JArray>()
.SelectMany(a => a.Ancestors().Where(p => p.Type != JTokenType.Property)));
return container.DescendantsAndSelf().Where(c => c.Parent is JArray && !nonLeafEntries.Contains(c));
}
}
Then put the returned items in an array of their own with:
var leafItemArray = new JArray(rootJContainer.LeafArrayEntries());
Related
How can I list all my "children" elements recursively using the select token method?
Actual JSON Structure
{
"component": "stackpanel",
"id": "001",
"children": [
{
"component": "label",
"id": "001.1",
"text": "foobar123",
"children": []
},
{
"component": "textbox",
"id": "001.2",
"text": "",
"children": []
},
{
"component": "stackpanel",
"id": "001.3",
"children": [
{
"component": "textbox",
"id": "001.3.1",
"text": "subfoobar",
"children": []
}
]
}
]
}
My code Implementation
var listChildrenElements= jsonElement.SelectTokens("$..*").OfType<JObject>();
Result I wait for the select token:
[
{
"component": "stackpanel",
"id": "001",
"children": [...]
},
{
"component": "label",
"id": "001.1",
"text": "foobar123",
"children": []
},
{
"component": "textbox",
"id": "001.2",
"text": "",
"children": []
},
{
"component": "stackpanel",
"id": "001.3",
"children": [...]
},
{
"component": "textbox",
"id": "001.3.1",
"text": "subfoobar",
"children": []
}
]
Is it possible to make the list according to the expected model example that I hope to receive? This in a recursive way because there can be numerous elements within the "children"
Try this:
public static IEnumerable<JObject> SelectHierarchy(IEnumerable<JObject> source)
{
foreach (var item in source)
{
yield return item;
var childResults = SelectHierarchy(item["children"].Children().Cast<JObject>());
foreach (var childItem in childResults)
{
yield return childItem;
}
}
}
Then call it on IEnumerable with your root item:
var list = SelectHierarchy(new [] {jobject});
I have a JArray of JArrays, but I would like to flatten it into a single JArray of JObjects. I have already implemented a foreach loop which iterates through each JArray in my JArray. I need to know how to flatten each sub-JArray into a JObject.
Here is an example:
[
{
"item": [
{
"fieldName": "Name",
"value": "Andy"
},
{
"fieldName": "Phone",
"value": "678-905-9872"
}
]
},
{
"item": [
{
"fieldName": "Name",
"value": "John"
},
{
"fieldName": "Phone",
"value": "688-954-5678"
}
]
},
{
"item": [
{
"fieldName": "Name",
"value": "Ashley"
},
{
"fieldName": "Phone",
"value": "+44 671 542 8945"
}
]
},
{
"item": [
{
"fieldName": "Name",
"value": "Avi"
},
{
"fieldName": "Phone",
"value": "(212)-908-7772"
}
]
}
]
I would like each item to be a single JObject, resulting in the following JArray:
[
{
"Name": "Andy"
"Phone": "678-905-9872"
},
{
"Name": "John"
"Phone": "688-954-5678"
{
"Name": "Ashley"
"Phone": "+44 671 542 8945"
},
{
"Name": "Avi"
"Phone": "(212)-908-7772"
}
]
Thanks!
EDIT
Here is my solution (c#, using Newtonsoft.Json)
public string ParserFunction(string json)
{
string fieldname, fieldvalue;
JArray jsonArray = JArray.Parse(json);
foreach (JObject item in jsonArray)
{
JArray temp = (JArray)item["columns"]; //create new temporary JArray
foreach (JObject jobject in temp)
{
fieldname = jobject["fieldName"].ToString();
fieldvalue = jobject["value"].ToString();
item.Add(fieldname, fieldvalue);
jobject.Remove("fieldName");
jobject.Remove("value");
}
item.Remove("item");
}
json = jsonArray.ToString();
return json;
}
Not sure if this is the most optimal way to do it, I saw an answer below which looks alright as well.
var jArr = new JArray(JArray.Parse(JSON)
.Select(x => new JObject(new JProperty("Name", x["item"][0]["Name"]),
new JProperty("Phone", x["item"][1]["Phone"])
)
)
);
var str = JsonConvert.SerializeObject(jArr, Formatting.Indented);
str would be:
[
{
"Name": "Andy",
"Phone": "(785) 241-6200"
},
{
"Name": "Arthur Song",
"Phone": "(212) 842-5500"
},
{
"Name": "Ashley James",
"Phone": "+44 191 4956203"
},
{
"Name": "Avi Green",
"Phone": "(212) 842-5500"
}
]
I have a WebAPI method that returns Json in a flexible structure that depends on the request.
Part of the problem is that there could be any number of columns, and they could be any type. The 2 given below (Code and Count) are just one example.
This structure is based on the underlying classes but there could be any number of columns in the output. So, rather than the usual properties you might expect, these are objects in a collection with Name and Value properties.
The downside of this flexible approach is that it gives a non-standard format.
Is there a way to transform this into a more normalised shape? Are there maybe some attributes I can add to the class properties to change the way they are serialised?
For example, where there are 2 columns - Code (string) and Count (numeric):
Current Json:
{
"Rows": [
{
"Columns": [
{
"Value": "1",
"Name": "Code"
},
{
"Value": 13,
"Name": "Count"
}
]
},
{
"Columns": [
{
"Value": "2",
"Name": "Code"
},
{
"Value": 12,
"Name": "Count"
}
]
},
{
"Columns": [
{
"Value": "9",
"Name": "Code"
},
{
"Value": 1,
"Name": "Count"
}
]
},
{
"Columns": [
{
"Value": "5",
"Name": "Code"
},
{
"Value": 2,
"Name": "Count"
}
]
}
]
}
Ideally I'd like to transform it to this:
{
"Rows": [
{
"Code": "1",
"Count": 13
},
{
"Code": "2",
"Count": 12
},
{
"Code": "9",
"Count": 1
},
{
"Code": "5",
"Count": 2
}
]
}
The controller method (C#)
public ReportResponse Get(ReportRequest request)
{
var result = ReportLogic.GetReport(request);
return result;
}
The output classes
public class ReportResponse
{
public List<ReportRow> Rows { get; set; }
public ReportResponse()
{
Rows = new List<ReportRow>();
}
}
public class ReportRow
{
public List<ReportColumn> Columns { get; set; }
public ReportRow()
{
Columns = new List<ReportColumn>();
}
}
public class ReportColumn<T> : ReportColumn
{
public T Value { get; set; }
public ReportColumn(string name)
{
Name = name;
}
}
public abstract class ReportColumn
{
public string Name { get; internal set; }
}
I think the easiest way would be to map your class to a dictionary before serializing. Something like:
var dictionaries = List<Dictionary<string, object>();
foreach(var column in rows.Columns)
{
dictionaries.Add(new Dictionary<string, object>{{column.Name, column.Value}});
}
Then serialize the dictionaries variable should do the trick.
If you're using the output in JavaScript, you could translate as follows:
var
data = {
"Rows": [
{
"Columns": [
{
"Value": "1",
"Name": "Code"
},
{
"Value": 13,
"Name": "Count"
}
]
},
{
"Columns": [
{
"Value": "2",
"Name": "Code"
},
{
"Value": 12,
"Name": "Count"
}
]
},
{
"Columns": [
{
"Value": "9",
"Name": "Code"
},
{
"Value": 1,
"Name": "Count"
}
]
},
{
"Columns": [
{
"Value": "5",
"Name": "Code"
},
{
"Value": 2,
"Name": "Count"
}
]
}
]
},
output = [
];
data.Rows.forEach(function (row)
{
var
newRow = {};
row.Columns.forEach(function (column)
{
newRow[column.Name] = column.Value;
});
output.push(newRow);
})
console.log(JSON.stringify(output));
I have lots of product typed documents saved in elasticsearch but couldn't search by documents property values and need help.
//Product Class
public Guid productID { get; set; }
public string productName { get; set; }
public Guid productTypeID { get; set; }
public List<Guid> categoryIds {get; set; }
I'm trying to search like this:
//Search function
var esconn = Yaziylabir.Bukalemun.DataObjects.ElasticSearchConnectionManager.GetClient();
QueryContainer q = null;
if (!ProductType.HasValue || (ProductType.HasValue && ProductType.Value == B2CVariables.PRODUCTTYPE_PRODUCT))
{
q = Query<ProductModel>.Term(t => t.ProductTypeID, B2CVariables.PRODUCTTYPE_PRODUCT);
}
if (Category != null)
{
//catListZ is also List<Guid>
q &= Query<ProductModel>.Terms(u=>u.Field(z=>z.CategoryIDs).Terms<Guid>(catListZ));
}
// as a bonus I also have keyword search
if (!string.IsNullOrEmpty(Keyword))
{
q &= Query<ProductModel>.QueryString(t => t.Query(Keyword));
}
//do search
var pp = new SearchRequest(Yaziylabir.Bukalemun.DataObjects.ElasticSearchConnectionManager.DefaultIndex, "product");
pp.Query = q;
pp.Size = PageSize;
var res = esconn.Search<ProductModel>(pp);
rtnVal = res.Documents.ToList();
Now, I tried combinations of these (only producttypeID, only categoryIDs, only keyword, etc...) and watch what is happening with fiddler.
No result comes back, no errors are raised. Only 0 hits. Request body seems ok too.
When I check documents stored in that index, they are there and they have values required and should return in result.
What is wrong here? Do you have any ideas? Please help me here. I'm feeling ashamed to be the guy who couldn't search a database properly.
EDIT:
Search's body text:
{"size":12,"query":{"term":{"productTypeID":{"value":"cae344cf-8cfa-4960-8387-8ee89899c53f"}}}}
Example document:
{
"productID": "5687b8ac-c3fe-4f1a-9643-08b0bf6cede8",
"productName": "7011 Yelek",
"productCode": "701102Y001 ",
"currency": {
"currencyID": 1,
"sign": "TL",
"rate": 0
},
"normalPrice": 170,
"currentPrice": 84.9,
"isDiscounted": true,
"taxRate": 8,
"productTypeID": "cae344cf-8cfa-4960-8387-8ee89899c53f",
"defaultImagePath": "/contents/images/productimages/75681ee4-19b3-4c7d-a24b-b3566085a980.jpg",
"totalStockCount": 8,
"totalStockRecordCount": 4,
"isInStock": true,
"statusID": "9ad17471-2ff2-4eb0-9cb0-4b86922263ea",
"categoryIDs": [
"a8c83f54-b784-4866-89c3-cabc641490d5",
"9d5a9ab7-8edb-4d5a-800b-c48bf6575d78"
]
}
I didn't include all properties because it will make document very long.
Here is the mapping:
{
"mappings": {
"product": {
"properties": {
"categoryIDs": {
"type": "string"
},
"currentPrice": {
"type": "double"
},
"isDiscounted": {
"type": "boolean"
},
"isInStock": {
"type": "boolean"
},
"normalPrice": {
"type": "double"
},
"productCode": {
"type": "string"
},
"productID": {
"type": "string"
},
"productName": {
"type": "string"
},
"productTypeID": {
"type": "string"
},
"statusID": {
"type": "string"
},
"taxRate": {
"type": "double"
},
"totalStockCount": {
"type": "long"
},
"totalStockRecordCount": {
"type": "long"
}
}
}
}
}
I suspect the productTypeID field is using either the default analyzer - standard - or any other analyzer that's splitting it in the wrong places. What you need is for productTypeID to be index: not_analyzed or analyzed with something like keyword. And you need to create the mapping manually, otherwise you can't do it the way you want it.
The idea is that ES is tokenizing by default your productTypeID values and will split them at -, so in the index you'd have tokens not the entire value. You, instead, need to have that value unchanged so that term will match it fully.
For example, not to mess up with your previous mapping, you can add fields to define a sub-field that will be not_analyzed:
"productTypeID": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
And your query needs to change slightly: {"size":12,"query":{"term":{"productTypeID.raw":{"value":"cae344cf-8cfa-4960-8387-8ee89899c53f"}}}}
For completeness sake, this is the complete command to create the mapping by hand. You could do it while the index is still "alive", but you'd still have to re-index the documents:
curl -XPUT "http://localhost:9200/your_index" -d'
{
"mappings": {
"product": {
"properties": {
"categoryIDs": {
"type": "string"
},
"currentPrice": {
"type": "double"
},
"isDiscounted": {
"type": "boolean"
},
"isInStock": {
"type": "boolean"
},
"normalPrice": {
"type": "double"
},
"productCode": {
"type": "string"
},
"productID": {
"type": "string"
},
"productName": {
"type": "string"
},
"productTypeID": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"statusID": {
"type": "string"
},
"taxRate": {
"type": "double"
},
"totalStockCount": {
"type": "long"
},
"totalStockRecordCount": {
"type": "long"
}
}
}
}
}'
I'm trying to use NEST to create an index with raw json and it produces different results than when I use that same json string interactively against elastic search. Is this a bug or am I using it incorrectly?
Posting directly to elastic search with the following command I get exactly the mappings I want for my index (results shown below)
POST /entities
{
"mappings": {
"sis_item" :
{
"properties":
{
"FullPath":
{
"type": "string",
"index":"not_analyzed"
},
"Ident":
{
"type": "nested",
"properties":
{
"ObjectGuid":
{
"type": "string",
"index":"not_analyzed"
}
}
}
}
}
}
Result when I check the index with: GET /entities/ : (which is correct)
{
"entities": {
"aliases": {},
"mappings": {
"sis_item": {
"properties": {
"FullPath": {
"type": "string",
"index": "not_analyzed"
},
"Ident": {
"type": "nested",
"properties": {
"ObjectGuid": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
},
"settings": {
"index": {
"creation_date": "1453828294488",
"number_of_shards": "5",
"number_of_replicas": "1",
"version": {
"created": "1070499"
},
"uuid": "6_j4vRcqRwiTQw0E6bQokQ"
}
},
"warmers": {}
}
}
However I have to do this from code and using the following code the mappings I specify end up in the settings instead of the mappings as shown below.
var createIndexJson = #"
{
""mappings"": {
""sis_item"" :
{
""properties"":
{
""FullPath"":
{
""type"": ""string"",
""index"":""not_analyzed""
},
""Ident"":
{
""type"": ""nested"",
""properties"":
{
""ObjectGuid"":
{
""type"": ""string"",
""index"":""not_analyzed""
}
}
}
}
}
}
}";
var response = _client.Raw.IndicesCreatePost("entities_from_code", createIndexJson);
if (!response.Success || response.HttpStatusCode != 200)
{
throw new ElasticsearchServerException(response.ServerError);
}
Result (not correct, notice the mappings are nested inside the settings):
{
"entities_from_code": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"mappings": {
"sis_item": {
"properties": {
"FullPath": {
"type": "string",
"index": "not_analyzed"
},
"Ident": {
"type": "nested",
"properties": {
"ObjectGuid": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
},
"creation_date": "1453828283162",
"number_of_shards": "5",
"number_of_replicas": "1",
"version": {
"created": "1070499"
},
"uuid": "fdmPqahGToCJw_mIbq0kNw"
}
},
"warmers": {}
}
}
There is a newline at the very top of the json string which cause the odd result, removing it gave me the expected behaviour.