I'd like to use moreLikeThis query on Elasticsearch using NEST library and give different boost values for each match.
var moreLikeThis = _elastic.Search<Report>(s => s
.From(0)
.Size(10)
.Query(q => q
.Filtered(f => f
.Query(fq => fq
.Dismax(dmx => dmx
.TieBreaker(0.7)
.Queries(qr => qr
.MoreLikeThis(mlt => mlt
.OnFields(of => of.Title.Suffix("stemmed"))
.MinTermFrequency(1)
.MaxQueryTerms(12)
.Boost(20)
.Documents(docs => docs
.Document() // This is the part where I'm stuck
)
), qr => qr
.MoreLikeThis(mlt => mlt
.OnFields(of => of.Title.Suffix("synonym"))
.MinTermFrequency(1)
.MaxQueryTerms(12)
.Boost(10)
)
)
)
)
)
)
);
This the query I'm stuck at. I can write this easily in raw JSON format, but that's not what I'm aiming for. I'm quite new in both C# and NEST and do not know how to pass documentId there.
That's my class, if it helps at all :
[ElasticType(IdProperty = "_id", Name = "reports")]
public class Report
{
[ElasticProperty(Name = "_id", Type = FieldType.String)]
public string _id { get; set; }
[ElasticProperty(Name = "Title", Type = FieldType.String)]
public string Title { get; set; }
}
And that's the query I'm using as JSON and it works just fine.
{
"from": 0,
"size": 10,
"fields": ["Title"],
"query": {
"filtered": {
"query": {
"dis_max": {
"tie_breaker": 0.7,
"queries": [
{
"more_like_this": {
"fields": ["Title.stemmed"],
"docs": [
{
"_index": "test",
"_type": "reports",
"_id": "68753"
}
],
"min_term_freq": 1,
"max_query_terms": 12
}
}, {
"more_like_this": {
"fields": ["Title.synonym"],
"docs": [
{
"_index": "test",
"_type": "reports",
"_id": "68753"
}
],
"min_term_freq": 1,
"max_query_terms": 12
}
}
]
}
}
}
}
}
Documentation in NEST doesn't explain or give a basic idea how this is done.
Thank you.
As stated in comments this was a bug and will be fixed together with 1.5.2 NEST release.
Related
I have 3 levels of JSON object and trying to sort based on one of the inner-element.
Here is the sample JSON.
public class Program
{
public static void Main()
{
string myJSON = #"
{
""items"": ""2"",
""documents"": [
{
""document"": {
""libraryId"": ""LIB0001"",
""id"": ""100"",
""elements"": {
""heading"": {
""elementType"": ""text"",
""value"": ""My Heading 1""
},
""date"": {
""elementType"": ""datetime"",
""value"": ""2020-07-03T20:30:00-04:00""
}
},
""name"": ""My Name 1 "",
""typeId"": ""10ed9f3f-ab41-45a9-ba24-d988974affa7""
}
},
{
""document"": {
""libraryId"": ""LIB0001"",
""id"": ""101"",
""elements"": {
""heading"": {
""elementType"": ""text"",
""value"": ""My Heading 2""
},
""date"": {
""elementType"": ""datetime"",
""value"": ""2020-07-03T20:30:00-04:00""
}
},
""name"": ""My Name 2"",
""typeId"": ""10ed9f3f-ab41-45a9-ba24-d988974affa7""
}
}
]
}";
JObject resultObject = JObject.Parse(myJSON);
var sortedObj = new JObject(
resultObject.Properties().OrderByDescending(p => p.Value)
);
string output = sortedObj.ToString();
Console.WriteLine(output);
}
}
I would like to sort based the "date" field. Appreciate any help.
You can replace documents json array with sorted one using json path to sort it:
resultObject["documents"] = new JArray(resultObject["documents"]
.Children()
.OrderBy(p => p.SelectToken("$.document.elements.date.value").Value<DateTime>()));
Console.WriteLine(resultObject.ToString());
Or using indexer access:
resultObject["documents"] = new JArray( resultObject["documents"]
.Children()
.OrderBy(p => p["document"]["elements"]["date"]["value"].Value<DateTime>()));
I've a sample company JSON document structure like this:
[
{
"id": "id1",
"name": "company1",
"employees": [
{
"name": "employee1",
"education": "education1"
},
{
"name": "employee2",
"education": "education2"
}
]
}
]
and I'm doing queries like this:
GET companies/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"employees.education": {
"value": "education1"
}
}
},
{
"term": {
"employees.education": {
"value": "education2"
}
}
}
]
}
}
}
The query is build using NEST:
var filters = new List<Func<QueryContainerDescriptor<Company>, QueryContainer>>();
foreach (var education in educationsToFilter)
{
filters.Add(fq => fq.Term(f => f.Employees.Suffix("education"), education));
}
var searchResponse = _client.Search<Company>(s => s
.Query(q => q
.Bool(bq => bq
.Filter(filters)
)
)
);
Is there a better way of finding the Term Field instead of using the Suffix-method? I would like a more type safe way.
You can resolve nested field name with this lambda expression:
fq.Term(f => f.Employees.FirstOrDefault().Education, education)
Hope that helps.
I would like to create an Aggregation on my data to get the total amount of counts for specific tags for a collection of books in my .Net application.
I have the following Book class.
public class Book
{
public string Id { get; set; }
public string Name { get; set; }
[BsonDictionaryOptions(DictionaryRepresentation.Document)]
public Dictionary<string, string> Tags { get; set; }
}
And when the data is saved, it is stored in the following format in MongoDB.
{
"_id" : ObjectId("574325a36fdc967af03766dc"),
"Name" : "My First Book",
"Tags" : {
"Edition" : "First",
"Type" : "HardBack",
"Published" : "2017",
}
}
I've been using facets directly in MongoDB and I am able to get the results that I need by using the following query:
db.{myCollection}.aggregate(
[
{
$match: {
"Name" : "SearchValue"
}
},
{
$facet: {
"categorizedByTags" : [
{
$project :
{
Tags: { $objectToArray: "$Tags" }
}
},
{ $unwind : "$Tags"},
{ $sortByCount : "$Tags"}
]
}
},
]
);
However I am unable to transfer this over to the .NET C# Driver for Mongo. How can I do this using the .NET C# driver?
Edit - I will ultimately be looking to query the DB on other properties of the books as part of a faceted book listings page, such as Publisher, Author, Page count etc... hence the usage of $facet, unless there is a better way of doing this?
I would personally not use $facet here since you've only got one pipeline which kind of defeats the purpose of $facet in the first place...
The following is simpler and scales better ($facet will create one single potentially massive document).
db.collection.aggregate([
{
$match: {
"Name" : "My First Book"
}
}, {
$project: {
"Tags": {
$objectToArray: "$Tags"
}
}
}, {
$unwind: "$Tags"
}, {
$sortByCount: "$Tags"
}, {
$group: { // not really needed unless you need to have all results in one single document
"_id": null,
"categorizedByTags": {
$push: "$$ROOT"
}
}
}, {
$project: { // not really needed, either: remove _id field
"_id": 0
}
}])
This could be written using the C# driver as follows:
var collection = new MongoClient().GetDatabase("test").GetCollection<Book>("test");
var pipeline = collection.Aggregate()
.Match(b => b.Name == "My First Book")
.Project("{Tags: { $objectToArray: \"$Tags\" }}")
.Unwind("Tags")
.SortByCount<BsonDocument>("$Tags");
var output = pipeline.ToList().ToJson(new JsonWriterSettings {Indent = true});
Console.WriteLine(output);
Here's the version using a facet:
var collection = new MongoClient().GetDatabase("test").GetCollection<Book>("test");
var project = PipelineStageDefinitionBuilder.Project<Book, BsonDocument>("{Tags: { $objectToArray: \"$Tags\" }}");
var unwind = PipelineStageDefinitionBuilder.Unwind<BsonDocument, BsonDocument>("Tags");
var sortByCount = PipelineStageDefinitionBuilder.SortByCount<BsonDocument, BsonDocument>("$Tags");
var pipeline = PipelineDefinition<Book, AggregateSortByCountResult<BsonDocument>>.Create(new IPipelineStageDefinition[] { project, unwind, sortByCount });
// string based alternative version
//var pipeline = PipelineDefinition<Book, BsonDocument>.Create(
// "{ $project :{ Tags: { $objectToArray: \"$Tags\" } } }",
// "{ $unwind : \"$Tags\" }",
// "{ $sortByCount : \"$Tags\" }");
var facetPipeline = AggregateFacet.Create("categorizedByTags", pipeline);
var aggregation = collection.Aggregate().Match(b => b.Name == "My First Book").Facet(facetPipeline);
var output = aggregation.Single().Facets.ToJson(new JsonWriterSettings { Indent = true });
Console.WriteLine(output);
I have the following JSON, and I want to take a list of IDs with its values
For example in LINQ something like this: result.Select(x=>x.id)
I tried this:
var inner = outer["pictures"].Value<JArray>().ToList();
I have a list there but with all attributes but I am not able to select just IDs since it is an anonymous list.
outer
{{
"id": "669654603",
"pictures": [
{
"id": "659745-MLA25600661898_052017",
"url": "http://mla-s2-p.mlstatic.com/659745-MLA25600661898_052017-O.jpg",
"secure_url": "https://mla-s2-p.mlstatic.com/659745-MLA25600661898_052017-O.jpg",
"size": "500x365",
"max_size": "625x457",
"quality": ""
},
{
"id": "908422-MLA25658267858_062017",
"url": "http://mla-s2-p.mlstatic.com/908422-MLA25658267858_062017-O.jpg",
"secure_url": "https://mla-s2-p.mlstatic.com/908422-MLA25658267858_062017-O.jpg",
"size": "47x47",
"max_size": "47x47",
"quality": ""
},
{
"id": "794138-MLA25658267889_062017",
"url": "http://mla-s2-p.mlstatic.com/794138-MLA25658267889_062017-O.jpg",
"secure_url": "https://mla-s2-p.mlstatic.com/794138-MLA25658267889_062017-O.jpg",
"size": "40x40",
"max_size": "40x40",
"quality": ""
}
]
}}
When working with Newtonsoft JSON you can do it like this:
var values = JObject.Parse(jsonString)["pictures"].Select(p => p["id"].Value<string>()).ToList();
In order to be valid your json should not contain double curly brackets:
{
"id": "669654603",
"pictures": [ ... ]
}
You can parse it without additional classes:
var ids = JObject.Parse(json)["pictures"].Select(p => (string)p["id"]);
Output:
[
"659745-MLA25600661898_052017",
"908422-MLA25658267858_062017",
"794138-MLA25658267889_062017"
]
Or you can create several classes to hold your json data:
public class Container
{
public List<Picture> Pictures { get; set; }
}
public class Picture
{
public string Id { get; set; }
// you can add other properties here
}
And use strongly-typed parsing:
var ids = JsonConvert.DeserializeObject<Container>(json).Pictures.Select(p => p.Id);
I've got a JSON stream coming back from a server, and I need to search for a specific value of the node "ID" using JSON.net to parse the data.
And I can almost make it work, but not quite because the results coming back are deeply nested in each other -- this is due to the fact that I'm getting a folder structure back. I've boiled the JSON down to a much simpler version. I'm getting this:
{
"data": {
"id": 0,
"name": "",
"childFolders": [{
"id": 19002,
"name": "Locker",
"childFolders": [{
"id": 19003,
"name": "Folder1",
"childFolders": [],
"childComponents": [{
"id": 19005,
"name": "route1",
"state": "STOPPED",
"type": "ROUTE"
}]
}, {
"id": 19004,
"name": "Folder2",
"childFolders": [],
"childComponents": [{
"id": 19008,
"name": "comm1",
"state": "STOPPED",
"type": "COMMUNICATION_POINT"
}, {
"id": 19006,
"name": "route2",
"state": "STOPPED",
"type": "ROUTE"
}, {
"id": 19007,
"name": "route3",
"state": "STOPPED",
"type": "ROUTE"
}]
}],
"childComponents": []
}],
"childComponents": []
},
"error": null
}
I can almost get there by going:
var objects = JObject.Parse(results);
var subobjects = objects["data"]["childFolders"][0]["childFolders"][1];
I can see in the debug view that it'll parse the object, but won't let me search within.
My ultimate goal is to be able to search for "route3" and get back 19007, since that's the ID for that route. I've found some results, but all of them assume you know how far nested the object is. The object I'm searching for could be 2 deep or 20 deep.
My ultimate goal is to be able to search for "route3" and get back 19007
You can use linq and Descendants method of JObject to do it:
var dirs = JObject.Parse(json)
.Descendants()
.Where(x=>x is JObject)
.Where(x=>x["id"]!=null && x["name"]!=null)
.Select(x =>new { ID= (int)x["id"], Name = (string)x["name"] })
.ToList();
var id = dirs.Find(x => x.Name == "route3").ID;
You can use the SelectToken or SelectTokens functions to provide a JPath to search for your desired node. Here is an example that would provide you the route based on name:
JObject.Parse(jsonData)["data"].SelectToken("$..childComponents[?(#.name=='route3')]")
You can find more documentation on JPath here
Simply write a recursive function:
private Thing FindThing(Thing thing, string name)
{
if (thing.name == name)
return thing;
foreach (var subThing in thing.childFolders.Concat(thing.childComponents))
{
var foundSub = FindThing(subThing, name);
if (foundSub != null)
return foundSub;
}
return null;
}
class RootObject
{
public Thing data { get; set; }
}
class Thing
{
public int id { get; set; }
public string name { get; set; }
public List<Thing> childFolders { get; set; } = new List<Thing>();
public List<Thing> childComponents { get; set; } = new List<Thing>();
}
And using it:
var obj = JsonConvert.DeserializeObject<RootObject>(jsonString);
var result = FindThing(obj.data, "route3");