I've a sample company JSON document structure like this:
[
{
"id": "id1",
"name": "company1",
"employees": [
{
"name": "employee1",
"education": "education1"
},
{
"name": "employee2",
"education": "education2"
}
]
}
]
and I'm doing queries like this:
GET companies/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"employees.education": {
"value": "education1"
}
}
},
{
"term": {
"employees.education": {
"value": "education2"
}
}
}
]
}
}
}
The query is build using NEST:
var filters = new List<Func<QueryContainerDescriptor<Company>, QueryContainer>>();
foreach (var education in educationsToFilter)
{
filters.Add(fq => fq.Term(f => f.Employees.Suffix("education"), education));
}
var searchResponse = _client.Search<Company>(s => s
.Query(q => q
.Bool(bq => bq
.Filter(filters)
)
)
);
Is there a better way of finding the Term Field instead of using the Suffix-method? I would like a more type safe way.
You can resolve nested field name with this lambda expression:
fq.Term(f => f.Employees.FirstOrDefault().Education, education)
Hope that helps.
Related
I have a doubt if it is possible to serialize a collection of BsonDocument results as a JSON pair of key objects.
For example, I attach a piece of code that creates a collection of BsonDocument with and _id and name as fields,
using MongoDB.Bson;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace ExampleBsonDocumentSerializeToJsonAsArray
{
class Program
{
static void Main(string[] args)
{
BsonDocument[] data = new BsonDocument[]{
new BsonDocument {
{ "_id" , "1" },
{ "name" , "name_1" },
{ "description" , "description_1" },
}
,new BsonDocument {
{ "_id" , "2" },
{ "name" , "name_2" },
{ "description" , "description_2" },
}
,new BsonDocument {
{ "_id" , "3" },
{ "name" , "name_3" },
{ "description" , "description_3" },
}
};
Console.WriteLine(data.ToJson());
}
}
}
List 1.1
The piece from the list 1.1 shows it gives output as a JSON array of objects:
[{
"_id": "1",
"name": "name_1",
"description": "description_1"
}, {
"_id": "2",
"name": "name_2",
"description": "description_2"
}, {
"_id": "3",
"name": "name_3",
"description": "description_3"
}]
Having the field '_id' as the key of the collection, I would like to serialize it as a set of key-object JSON instead of an array of objects. The result of serialized JSON should be like this,
{
"1": {
"name": "name_1"
, "description": "description_1"
},
"2": {
"name": "name_2"
, "description": "description_2"
},
"3": {
"name": "name_3"
, "description": "description_3"
}
}
I don't know whether is this possible or not.
You can convert BsonDocument to Dictionary via System.Linq.
using System.Linq;
var kvp = data.AsEnumerable()
.ToDictionary(x => x["_id"], x => new { name = x["name"], description = x["description"] });
Console.WriteLine(kvp.ToJson());
Sample program
I have 3 levels of JSON object and trying to sort based on one of the inner-element.
Here is the sample JSON.
public class Program
{
public static void Main()
{
string myJSON = #"
{
""items"": ""2"",
""documents"": [
{
""document"": {
""libraryId"": ""LIB0001"",
""id"": ""100"",
""elements"": {
""heading"": {
""elementType"": ""text"",
""value"": ""My Heading 1""
},
""date"": {
""elementType"": ""datetime"",
""value"": ""2020-07-03T20:30:00-04:00""
}
},
""name"": ""My Name 1 "",
""typeId"": ""10ed9f3f-ab41-45a9-ba24-d988974affa7""
}
},
{
""document"": {
""libraryId"": ""LIB0001"",
""id"": ""101"",
""elements"": {
""heading"": {
""elementType"": ""text"",
""value"": ""My Heading 2""
},
""date"": {
""elementType"": ""datetime"",
""value"": ""2020-07-03T20:30:00-04:00""
}
},
""name"": ""My Name 2"",
""typeId"": ""10ed9f3f-ab41-45a9-ba24-d988974affa7""
}
}
]
}";
JObject resultObject = JObject.Parse(myJSON);
var sortedObj = new JObject(
resultObject.Properties().OrderByDescending(p => p.Value)
);
string output = sortedObj.ToString();
Console.WriteLine(output);
}
}
I would like to sort based the "date" field. Appreciate any help.
You can replace documents json array with sorted one using json path to sort it:
resultObject["documents"] = new JArray(resultObject["documents"]
.Children()
.OrderBy(p => p.SelectToken("$.document.elements.date.value").Value<DateTime>()));
Console.WriteLine(resultObject.ToString());
Or using indexer access:
resultObject["documents"] = new JArray( resultObject["documents"]
.Children()
.OrderBy(p => p["document"]["elements"]["date"]["value"].Value<DateTime>()));
I am working on a POC to include Search with ElasticSearch(v7.3) in C# using ElasticSearch.Net and Nest clients. I am trying to build a Json query into Nest Query DSL to get some Data.
Here is the Json Query:
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "human",
"analyzer": "standard",
"type": "most_fields",
"fields": [
"hasParts.levelProperties.pageTranscript^100",
"contentTitle^90",
"documentTitle^80",
"pageTranscript^70"
]
}
}
],
"filter": {
"bool": {
"must": [
{ "term" : { "documentLevel" : 1 } }
],
"must_not": [
{ "term" : { "moduleNumber" : 4 } }
],
"should" : [
{
"bool" :{
"must" : [
{ "term" : { "coverDateStartSpecified" : true } },
{ "term" : { "coverDateEndSpecified" : true } },
{
"bool" : {
"should" : [
{ "range" : { "coverDateStartYear" : { "gte" : 1946, "lte" : 1975 } } },
{ "range" : { "coverDateEndYear" : { "gte" : 1946, "lte" : 1975 } } }
]
}
}
]
}
},
{
"bool" :{
"must" : [
{ "range" : { "coverDateYear" : { "gte" : 1946, "lte" : 1975 } } },
{
"bool" : {
"should" : [
{ "term" : { "coverDateStartSpecified" : false } },
{ "term" : { "coverDateEndSpecified" : false } }
]
}
}
]
}
}
]
}
}
}
}
}
This is a very complex query but this is just normal for the Project that I am working on.
I have tried to convert the query:
var responsedata = _connectionToEs.EsClient().Search<CrSearchContract>(s => s
.Size(100).Scroll(1).Query(q => q
.Bool(b => b
.Must(m => m
.MultiMatch(mm => mm
.Query("human")
.Analyzer("standard")
.Type(TextQueryType.MostFields)
.Fields(f => f.Field(ff => ff.DocumentTitle, 80)
.Field(ff => ff.contentTitle, 90)
.Field(ff => ff.PageTranscript, 70)
.Field(ff => ff.PublicationTitle, 60)
.Field(ff => ff.HasParts[0].LevelProperties.PageTranscript, 100)
)
)
)
.Filter(fil=>fil
.Bool(bl=>bl
.Must(fbm=>fbm.Term(ff=>ff.Field(p=>p.DocumentLevel).Value(1)))
.MustNot(fbmn=>fbmn.Term(ff=>ff.Field(p=>p.ModuleNumber).Value(4)))
.Should(fbs=>fbs
.Bool(fbsb=>fbsb
.Must(fbsbm=>fbsbm
.Term(ff => ff.Field(p => p.CoverDateStartSpecified).Value(true))
)
)
)
)
)
)
));
The issue I am facing is with converting the Filter Object of the JSON query to Query DSL. There are multiple Term query in each Must, Must_Not and Should Conditions, which when I try to add in the query gives an error QueryContainer doesn't contain a definition for 'Term' and no..... What am I doing wrong?
Thanks in Advance for the Help!!
You can do this like this
.Must(
fbm => fbm.Term(ff => ff.Field(p => p.DocumentLevel).Value(1)),
fbm => fbm.Term(ff => ff.Field(p => p.DocumentLevel).Value(2)))
but there are also a couple of other ways of writing bool queries in NEST. It's worth checking docs.
UPDATE
You can handle range query with following
.Range(r => r.Field(f => f.CoverDateStartYear).GreaterThanOrEquals(1946).LessThanOrEquals(1975))
Hope that helps.
I've got a JSON stream coming back from a server, and I need to search for a specific value of the node "ID" using JSON.net to parse the data.
And I can almost make it work, but not quite because the results coming back are deeply nested in each other -- this is due to the fact that I'm getting a folder structure back. I've boiled the JSON down to a much simpler version. I'm getting this:
{
"data": {
"id": 0,
"name": "",
"childFolders": [{
"id": 19002,
"name": "Locker",
"childFolders": [{
"id": 19003,
"name": "Folder1",
"childFolders": [],
"childComponents": [{
"id": 19005,
"name": "route1",
"state": "STOPPED",
"type": "ROUTE"
}]
}, {
"id": 19004,
"name": "Folder2",
"childFolders": [],
"childComponents": [{
"id": 19008,
"name": "comm1",
"state": "STOPPED",
"type": "COMMUNICATION_POINT"
}, {
"id": 19006,
"name": "route2",
"state": "STOPPED",
"type": "ROUTE"
}, {
"id": 19007,
"name": "route3",
"state": "STOPPED",
"type": "ROUTE"
}]
}],
"childComponents": []
}],
"childComponents": []
},
"error": null
}
I can almost get there by going:
var objects = JObject.Parse(results);
var subobjects = objects["data"]["childFolders"][0]["childFolders"][1];
I can see in the debug view that it'll parse the object, but won't let me search within.
My ultimate goal is to be able to search for "route3" and get back 19007, since that's the ID for that route. I've found some results, but all of them assume you know how far nested the object is. The object I'm searching for could be 2 deep or 20 deep.
My ultimate goal is to be able to search for "route3" and get back 19007
You can use linq and Descendants method of JObject to do it:
var dirs = JObject.Parse(json)
.Descendants()
.Where(x=>x is JObject)
.Where(x=>x["id"]!=null && x["name"]!=null)
.Select(x =>new { ID= (int)x["id"], Name = (string)x["name"] })
.ToList();
var id = dirs.Find(x => x.Name == "route3").ID;
You can use the SelectToken or SelectTokens functions to provide a JPath to search for your desired node. Here is an example that would provide you the route based on name:
JObject.Parse(jsonData)["data"].SelectToken("$..childComponents[?(#.name=='route3')]")
You can find more documentation on JPath here
Simply write a recursive function:
private Thing FindThing(Thing thing, string name)
{
if (thing.name == name)
return thing;
foreach (var subThing in thing.childFolders.Concat(thing.childComponents))
{
var foundSub = FindThing(subThing, name);
if (foundSub != null)
return foundSub;
}
return null;
}
class RootObject
{
public Thing data { get; set; }
}
class Thing
{
public int id { get; set; }
public string name { get; set; }
public List<Thing> childFolders { get; set; } = new List<Thing>();
public List<Thing> childComponents { get; set; } = new List<Thing>();
}
And using it:
var obj = JsonConvert.DeserializeObject<RootObject>(jsonString);
var result = FindThing(obj.data, "route3");
I'd like to use moreLikeThis query on Elasticsearch using NEST library and give different boost values for each match.
var moreLikeThis = _elastic.Search<Report>(s => s
.From(0)
.Size(10)
.Query(q => q
.Filtered(f => f
.Query(fq => fq
.Dismax(dmx => dmx
.TieBreaker(0.7)
.Queries(qr => qr
.MoreLikeThis(mlt => mlt
.OnFields(of => of.Title.Suffix("stemmed"))
.MinTermFrequency(1)
.MaxQueryTerms(12)
.Boost(20)
.Documents(docs => docs
.Document() // This is the part where I'm stuck
)
), qr => qr
.MoreLikeThis(mlt => mlt
.OnFields(of => of.Title.Suffix("synonym"))
.MinTermFrequency(1)
.MaxQueryTerms(12)
.Boost(10)
)
)
)
)
)
)
);
This the query I'm stuck at. I can write this easily in raw JSON format, but that's not what I'm aiming for. I'm quite new in both C# and NEST and do not know how to pass documentId there.
That's my class, if it helps at all :
[ElasticType(IdProperty = "_id", Name = "reports")]
public class Report
{
[ElasticProperty(Name = "_id", Type = FieldType.String)]
public string _id { get; set; }
[ElasticProperty(Name = "Title", Type = FieldType.String)]
public string Title { get; set; }
}
And that's the query I'm using as JSON and it works just fine.
{
"from": 0,
"size": 10,
"fields": ["Title"],
"query": {
"filtered": {
"query": {
"dis_max": {
"tie_breaker": 0.7,
"queries": [
{
"more_like_this": {
"fields": ["Title.stemmed"],
"docs": [
{
"_index": "test",
"_type": "reports",
"_id": "68753"
}
],
"min_term_freq": 1,
"max_query_terms": 12
}
}, {
"more_like_this": {
"fields": ["Title.synonym"],
"docs": [
{
"_index": "test",
"_type": "reports",
"_id": "68753"
}
],
"min_term_freq": 1,
"max_query_terms": 12
}
}
]
}
}
}
}
}
Documentation in NEST doesn't explain or give a basic idea how this is done.
Thank you.
As stated in comments this was a bug and will be fixed together with 1.5.2 NEST release.