Started working on NEST api for elastic search recently, got stuck on following query, the data.e would be dynamically populated using the input from client in the HttpGet,
ex: user sends eventA,eventB,eventC then we would add in the should part:
GET events/_search
{
"_source": false,
"query": {
"bool": {
"must": [
{"range": {
"timestamp": {
"gte": 1604684158527,
"lte": 1604684958731
}
}},
{"nested": {
"path": "data",
"query": {
"bool": {
"should": [
{"match": {
"data.e": "eventA"
}},
{"match": {
"data.e": "eventB"
}},
{"match": {
"data.e": "eventC"
}},
]
}
},
"inner_hits": {}
}}
]
}
}
}
Following is what I came up with till now:
var graphDataSearch = _esClient.Search<Events>(s => s
.Source(src => src
.Includes(i => i
.Field("timestamp")
)
)
.Query(q => q
.Bool(b => b
.Must(m => m
.Range(r => r
.Field("timestamp")
.GreaterThanOrEquals(startTime)
.LessThanOrEquals(stopTime)
),
m => m
.Nested(n => n
.Path("data")
.Query(q => q
.Bool(bo => bo
.Should(
// what to add here?
)
)
)
)
)
));
Can someone please help how to build the should part dynamically based on what input the user sends?
Thanks.
You can replace the nested query in the above snippet as shown below
// You may modify the parameters of this method as per your needs to reflect user input
// Field can be hardcoded as shown here or can be fetched from Event type as below
// m.Field(f => f.Data.e)
public static QueryContainer Blah(params string[] param)
{
return new QueryContainerDescriptor<Events>().Bool(
b => b.Should(
s => s.Match(m => m.Field("field1").Query(param[0])),
s => s.Match(m => m.Field("field2").Query(param[1])),
s => s.Match(m => m.Field("field3").Query(param[2]))));
}
What we are essentially doing here is we are returning a QueryContainer object that will be passed to the nested query
.Query(q => Blah(<your parameters>))
The same can be done by adding this inline without a separate method. You may choose which ever route you perfer. However, in general, having a method of its own increases the readability and keeps things cleaner.
You can read more about Match usage here
Edit:
Since you want to dynamically add the match queries inside this, below is a way you can do it.
private static QueryContainer[] InnerBlah(string field, string[] param)
{
QueryContainer orQuery = null;
List<QueryContainer> queryContainerList = new List<QueryContainer>();
foreach (var item in param)
{
orQuery = new MatchQuery() {Field = field, Query = item};
queryContainerList.Add(orQuery);
}
return queryContainerList.ToArray();
}
Now, call this method from inside of the above method as shown below
public static QueryContainer Blah(params string[] param)
{
return new QueryContainerDescriptor<Events>().Bool(
b => b.Should(
InnerBlah("field", param)));
}
Related
My Old DSL query from which i am getting 2237 records from same index and link
../design/_count?pretty=true
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "872.70344mi",
"location": {
"lat": 47.52,
"lon": -121.87
}
}
}
}
}
}
My NEST query by which I am getting 11093 records, total nine thousand records differance.
public async Task<long> GetEsDataCountByGeoDistanceAsync<T>(string indexName) where T : class
{
var searchResponse = await _elasticClient.CountAsync<T>(s => s
.Index(indexName)
.Query(q => q
.Bool(b => b
.Must(m => m
.MatchAll())
.Filter(f => f
.GeoDistance(go => go
.Distance("872.70344mi")
.Location(47.52, -121.87))
)))
).ConfigureAwait(false);
return searchResponse.Count;
}
so where I am missing some by which i am getting miss match data?
Thanks #Rob, I was tried with that syntax so I can saw actual DSL query behind above NEST query and solved by below query, actually was need to send field name for get geo distance..but dnt know why this is different with above DSL query..
public async Task<long> GetEsDataCountByGeoDistanceAsync<T>(string indexName,string fieldName, double latitude, double longtitude, string distance) where T : class
{
var searchResponse = await _elasticClient.CountAsync<T>(s => s
.Index(indexName)
.Query(q => q
.Bool(b => b
.Must(m => m
.MatchAll())
.Filter(f => f
.GeoDistance(go => go
.Field(fieldName)
.Distance(distance)
.Location(latitude, longtitude))
)))
).ConfigureAwait(false);
return searchResponse.Count;
}
When I search with Turkish characters in elasticsearch, it does not match. For example, when I type "yazilim", the result comes, but when I type "Yazılım", no result. The correct one is "Yazılım".
My index code.
var createIndexDescriptor = new CreateIndexDescriptor(INDEX_NAME).Mappings(ms => ms.Map<T>(m => m.AutoMap()
.Properties(pprops => pprops
.Text(ps => ps
.Name("Title")
.Fielddata(true)
.Fields(f => f
.Keyword(k => k
.Name("keyword")))))
)).Settings(st => st
.Analysis(an => an
.Analyzers(anz => anz
.Custom("tab_delim_analyzer", td => td
.Filters("lowercase", "asciifolding")
.Tokenizer("standard")
)
)
)
);
my search query code.
var searchResponse = eClient.Search<GlobalCompany>(s => s.Index(INDEX_NAME).From(0).Size(10)
.Query(q => q
.MultiMatch(m => m
.Fields(f => f
.Field(u => u.Title)
.Field(u => u.RegisterNumber))
.Type(TextQueryType.PhrasePrefix)
.Query(value))));
You are using an asciifolding filter, it makes sure ASCII characters are used (see docs).
You need to configure your field Title as a text field instead of a keyword field and set the analyzer for this field to tab_delim_analyzer.
I don't know how to translate this in dotNet world but here is what I mean in pure Kibana Dev Console script (curl):
DELETE deneme
PUT deneme
{
"settings": {
"analysis": {
"analyzer": {
"tab_delim_analyzer": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"asciifolding"
]
}
}
}
},
"mappings": {
"properties": {
"Title": {
"type": "text",
"analyzer": "tab_delim_analyzer"
}
}
}
}
I have about a hundred test documents in my index, built using NBuilder:
[
{
"title" : "Title1",
"text" : "Text1"
},
{
"title" : "Title2",
"text" : "Text2"
},
{
"title" : "Title3",
"text" : "Text3"
}
]
I want to query them with a wildcard to find all items with "text" starts with "Text". But when I use two wildcard methods in Nest I get two different results.
var response = await client.SearchAsync<FakeFile>(s => s.Query(q => q
.QueryString(d => d.Query("text:Text*")))
.From((page - 1) * pageSize)
.Size(pageSize));
This returns 100 results. But I'm trying to use a fluent API rather than querystring.
var response = await client.SearchAsync<FakeFile>(s => s
.Query(q => q
.Wildcard(c => c
.Field(f => f.Text)
.Value("Text*"))));
This returns 0 results. I'm new to Elasticsearch. I've tried to make the example as simple as possible to make sure I understand it piece-by-piece. I don't know why nothing is returning from the second query. Please help.
Assuming your text field is of type text, then during indexing elasticsearch will store Text1 as text1 internally in the inverted index. Exactly the same analysis will happen when using query string query, but not when you are using wildcard query.
.QueryString(d => d.Query("text:Text*"))) looks for text* and .Wildcard(c => c.Field(f => f.Text).Value("Text*"))) looks for Text* but elasticsearch stores internally only first one.
Hope that helps.
Supposed your mapping looks like that:
{
"mappings": {
"doc": {
"properties": {
"title": {
"type": "text"
},
"text":{
"type": "text"
}
}
}
}
}
Try this (Value should be in lowercase):
var response = await client.SearchAsync<FakeFile>(s => s
.Query(q => q
.Wildcard(c => c
.Field(f => f.Text)
.Value("text*"))));
Or this (don't know if f.Text has text property on it):
var response = await client.SearchAsync<FakeFile>(s => s
.Query(q => q
.Wildcard(c => c
.Field("text")
.Value("text*"))));
Kibana syntax:
GET index/_search
{
"query": {
"wildcard": {
"text": {
"value": "text*"
}
}
}
}
can you guide me as to how i can make C# equivalent
db.UserProfile.aggregate([
{$match:{_id:"sen"}},
{
$project: {
DemRole: {
$filter: {
input: "$DemRole",
as: "item",
cond: { $eq: [ "$$item.Name", "CO" ] }
}
}
}
}
])
I am trying to select a document if matches _id and retrieve the result after applying filter on embedded documents.It works fine in MongoDB on Robo3T. But i m not able to translate the same in C#.
This should get you going:
var collection = new MongoClient().GetDatabase("test").GetCollection<User>("UserProfile");
var pipeline = collection.Aggregate()
.Match(up => up.Id == "sen")
.Project(up => new { DemRole = up.DemRole.Where(c => c.Name == "CO") });
Thanks for the answer. It works in C# now. But Below is how i could get the information.
var pipeline = collection.Aggregate()
.Match(up => up.UserID == userId)
.Project(up => new { DemRole = up.DemRole.Where(c => c.Status== true) }).ToList();
// .Project(dem => dem.DemRole.Where(c => c.Status == true));//.ToList();
foreach(var pipe in pipeline)
{
return pipe.DemRole.ToList();
}
I would like to know the operation the below code performs
.Project(up => new { DemRole = up.DemRole.Where(c => c.Status== true) })
and why below code does not work.
.Project(dem => dem.DemRole.Where(c => c.Status == true));//.ToList();
Moreover i had to run a foreach to get the information as given below.
foreach(var pipe in pipeline)
{
return pipe.DemRole.ToList();
}
If you could explain the above lines or point me to some documentation which i can go through,it would be better.
I have a nested elasticsearch document and I want to search within all the fields of that document i.e I want to search in both the top-level and the nested fields. My index name is people and my type name is person.
My documents look like this :
{
"id": 1,
"fname": "elizabeth",
"mname": "nicolas",
"lname": "thomas",
"houseno": "beijing",
"car": [
{
"carname": "audi",
"carno": 4444,
"color": "black"
},
{
"carname": "mercedez",
"carno": 5555,
"color": "pink"
}
]
}
Then i have the following query in .net which actually searches for an user input keyword in the elasticsearch documents. Basically, I want to search in each and every field of a document. And I use inner_hits in my query so that i can return only the matching nested document.
I have designed my query as :
var result = client.Search<person>
(s => s
.From(from)
.Size(size)
.Source(false)
.Query(query => query.Filtered(filtered => filtered
.Query(q => q.MatchAll())
.Filter(f => f.Nested(nf => nf
.InnerHits()
.Path(p => p.car)
.Query(qq => qq.Match(m => m.OnField(g => g.car.First().carname).Query(searchKeyword))))))));
And my corresponding JSON query which i use in the head plugin is :
POST-people/person/_search:
{
"_source":false,
"query": {
"filtered": {
"query": {"match_all": {}},
"filter": {
"nested": {
"path": "car",
"filter": {
"term": {
"car.carname": "searchKeyword"
}
},
"inner_hits" : {}
}
}
}
}
}
But i wanted to search in all the fields(id,fname,mname,lname,houseno,carname,carno,color) and not just in a single field e.g. in carname as i have done in my above query.
Also, i want to do partial searching like %xyz%.
How can i do these ?
Can anyone help me modify this query so that i can use this single query to search within all the fields as well as do partial searching?
I'm new to elasticsearch as well as .net,so I would be thankful for any help.
Did you try using Query instead of OnField method?
I mean, having your query this way:
var result = client.Search<person>
(s => s
.From(from)
.Size(size)
.Source(false)
.Query(query => query.Filtered(filtered => filtered
.Query(q => q.MatchAll())
.Filter(f => f.Nested(nf => nf
.InnerHits()
.Path(p => p.car)
.Query(qq => qq.Match(m => m.Query(searchKeyword))))))));