I am trying to do a simple search in Elasticsearch.
in marvel, I am getting the results back using the following query:
GET /gl_search/poc/_search
{
"query": {
"term": {
"Account_No": "056109"
}
}
}
The type is poc and index is gl_search. See below
{
"took": 28,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 23586,
"max_score": 4.7722025,
"hits": [
{
"_index": "gl_search",
"_type": "poc",
"_id": "AU7fza_MU2-26YcvKIeM",
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(
node,
defaultIndex: "gl_search"
);
When using NEST, I am not getting any results back. I tried the following:
var r = client.Search<poc>(s => s
.From(0)
.Size(10)
.Query(q => q
.Term(p => p.Account_No, "056109")
)
and
var r = client.Search<poc>(query => query.Index("gl_search")
.Type("poc")
.From(0)
.Size(100)
.Filter(x => x.Term(p=> p.Journal_ID, "056109")));
Settings are:
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(
node,
defaultIndex: "gl_search"
);
Update
I dont see anything in Fiddler. I assume I would see something there. If so, is it the way I set it up or something kind of virus protection blocking it
As pointed out in the comments, NEST by default will camelcase .NET object property names when serializing the SearchDescriptor<T> into the query DSL. So, in your example above,
void Main()
{
var settings = new ConnectionSettings(new Uri("http://localhost:9200"));
var connection = new InMemoryConnection(settings);
var client = new ElasticClient(connection: connection);
var r = client.Search<poc>(s => s
.Index("gl_search")
.From(0)
.Size(10)
.Query(q => q
.Term(p => p.Account_No, "056109")
)
);
Console.WriteLine(Encoding.UTF8.GetString(r.RequestInformation.Request));
}
produces the following query DSL (notice the property is account_No)
{
"from": 0,
"size": 10,
"query": {
"term": {
"account_No": {
"value": "056109"
}
}
}
}
Now, there are two real options for addressing this
1.Use a string for the property name, bypassing NEST's camelcasing convention but at the expense of not working with lambda expressions and all of the type safety that they provide
var r = client.Search<poc>(s => s
.Index("gl_search")
.From(0)
.Size(10)
.Query(q => q
.Term("Account_No", "056109")
)
);
2.Change the default serialization options for NEST; useful if all of you properties are cased as per your examples
// change the default camelcase property name serialization
// behaviour with .SetDefaultPropertyNameInferrer
var settings = new ConnectionSettings(new Uri("http://localhost:9200"))
.SetDefaultPropertyNameInferrer(s => s);
var client = new ElasticClient(settings);
var r = client.Search<poc>(s => s
.Index("gl_search")
.From(0)
.Size(10)
.Query(q => q
.Term(p => p.Account_No, "056109")
)
);
A Func<string, string> is passed to .SetDefaultPropertyNameInferrer() that will be called for each property that will be serialized to form the json query DSL that is sent to Elasticsearch.
Related
Started working on NEST api for elastic search recently, got stuck on following query, the data.e would be dynamically populated using the input from client in the HttpGet,
ex: user sends eventA,eventB,eventC then we would add in the should part:
GET events/_search
{
"_source": false,
"query": {
"bool": {
"must": [
{"range": {
"timestamp": {
"gte": 1604684158527,
"lte": 1604684958731
}
}},
{"nested": {
"path": "data",
"query": {
"bool": {
"should": [
{"match": {
"data.e": "eventA"
}},
{"match": {
"data.e": "eventB"
}},
{"match": {
"data.e": "eventC"
}},
]
}
},
"inner_hits": {}
}}
]
}
}
}
Following is what I came up with till now:
var graphDataSearch = _esClient.Search<Events>(s => s
.Source(src => src
.Includes(i => i
.Field("timestamp")
)
)
.Query(q => q
.Bool(b => b
.Must(m => m
.Range(r => r
.Field("timestamp")
.GreaterThanOrEquals(startTime)
.LessThanOrEquals(stopTime)
),
m => m
.Nested(n => n
.Path("data")
.Query(q => q
.Bool(bo => bo
.Should(
// what to add here?
)
)
)
)
)
));
Can someone please help how to build the should part dynamically based on what input the user sends?
Thanks.
You can replace the nested query in the above snippet as shown below
// You may modify the parameters of this method as per your needs to reflect user input
// Field can be hardcoded as shown here or can be fetched from Event type as below
// m.Field(f => f.Data.e)
public static QueryContainer Blah(params string[] param)
{
return new QueryContainerDescriptor<Events>().Bool(
b => b.Should(
s => s.Match(m => m.Field("field1").Query(param[0])),
s => s.Match(m => m.Field("field2").Query(param[1])),
s => s.Match(m => m.Field("field3").Query(param[2]))));
}
What we are essentially doing here is we are returning a QueryContainer object that will be passed to the nested query
.Query(q => Blah(<your parameters>))
The same can be done by adding this inline without a separate method. You may choose which ever route you perfer. However, in general, having a method of its own increases the readability and keeps things cleaner.
You can read more about Match usage here
Edit:
Since you want to dynamically add the match queries inside this, below is a way you can do it.
private static QueryContainer[] InnerBlah(string field, string[] param)
{
QueryContainer orQuery = null;
List<QueryContainer> queryContainerList = new List<QueryContainer>();
foreach (var item in param)
{
orQuery = new MatchQuery() {Field = field, Query = item};
queryContainerList.Add(orQuery);
}
return queryContainerList.ToArray();
}
Now, call this method from inside of the above method as shown below
public static QueryContainer Blah(params string[] param)
{
return new QueryContainerDescriptor<Events>().Bool(
b => b.Should(
InnerBlah("field", param)));
}
I have about a hundred test documents in my index, built using NBuilder:
[
{
"title" : "Title1",
"text" : "Text1"
},
{
"title" : "Title2",
"text" : "Text2"
},
{
"title" : "Title3",
"text" : "Text3"
}
]
I want to query them with a wildcard to find all items with "text" starts with "Text". But when I use two wildcard methods in Nest I get two different results.
var response = await client.SearchAsync<FakeFile>(s => s.Query(q => q
.QueryString(d => d.Query("text:Text*")))
.From((page - 1) * pageSize)
.Size(pageSize));
This returns 100 results. But I'm trying to use a fluent API rather than querystring.
var response = await client.SearchAsync<FakeFile>(s => s
.Query(q => q
.Wildcard(c => c
.Field(f => f.Text)
.Value("Text*"))));
This returns 0 results. I'm new to Elasticsearch. I've tried to make the example as simple as possible to make sure I understand it piece-by-piece. I don't know why nothing is returning from the second query. Please help.
Assuming your text field is of type text, then during indexing elasticsearch will store Text1 as text1 internally in the inverted index. Exactly the same analysis will happen when using query string query, but not when you are using wildcard query.
.QueryString(d => d.Query("text:Text*"))) looks for text* and .Wildcard(c => c.Field(f => f.Text).Value("Text*"))) looks for Text* but elasticsearch stores internally only first one.
Hope that helps.
Supposed your mapping looks like that:
{
"mappings": {
"doc": {
"properties": {
"title": {
"type": "text"
},
"text":{
"type": "text"
}
}
}
}
}
Try this (Value should be in lowercase):
var response = await client.SearchAsync<FakeFile>(s => s
.Query(q => q
.Wildcard(c => c
.Field(f => f.Text)
.Value("text*"))));
Or this (don't know if f.Text has text property on it):
var response = await client.SearchAsync<FakeFile>(s => s
.Query(q => q
.Wildcard(c => c
.Field("text")
.Value("text*"))));
Kibana syntax:
GET index/_search
{
"query": {
"wildcard": {
"text": {
"value": "text*"
}
}
}
}
I am using Nest 6.2 with ES 6.6
I have the following code which runs fine:
var response = elasticClient.Search<PropertyRecord>(s => s
.Query(q => q.Terms(
c => c
.Field(f => f.property_id_orig)
.Terms(listOfPropertyIds) // a list of 20 ids say...
))
.From(0)
.Take(100) // just pull up to a 100...
);
if (!response.IsValid)
throw new Exception(response.ServerError.Error.Reason);
return response.Documents;
But I know there is an issue with the underlying query, because all documents from the index are returned. So I would like to be able to see the raw Json that is generated by the lambda expression so I can see the result running in the Head plugin or Fiddler etc.
If I use a SearchRequest Object and pass that to the Search method, would I then be able to see the Query Json?
var request = new SearchRequest
{
// and build equivalent query here
};
I am having trouble building the corresponding query using the SearchRequest approach and cannot find decent examples showing how to do so.
Anyone know?
You can get the JSON for any NEST request using SerializeToString extension method
var client = new ElasticClient();
var listOfPropertyIds = new [] { 1, 2, 3 };
// pull the descriptor out of the client API call
var searchDescriptor = new SearchDescriptor<PropertyRecord>()
.Query(q => q.Terms(
c => c
.Field(f => f.property_id_orig)
.Terms(listOfPropertyIds) // a list of 20 ids say...
))
.From(0)
.Take(100);
var json = client.RequestResponseSerializer.SerializeToString(searchDescriptor, SerializationFormatting.Indented);
Console.WriteLine(json);
which yields
{
"from": 0,
"query": {
"terms": {
"property_id_orig": [
1,
2,
3
]
}
},
"size": 100
}
this is my sql result
and i want to serialize the result to json
like following
{
"Studentid": 1000,
"ExamType":[
{
"Examtype":"TERM 2",
"ExamName":[{
"ExamName":"PERIODIC TEST 1-Term2",
"SubjectName": [{
"SubjectName":"SL-MALAYALAM",
"ComponentName":[{
"ComponenetName":"Exam",
"SubComponent":[{
"SubCOmponent":"Exam",
"ExamDate":"2017-08-03",
"MaxMark":"50.00",
"MarkObtained":"38.00",
"Grade":"B1"
}]
},
{
"ComponenetName":"NOTEBOOK",
"SubComponent":[{
"SubCOmponent":"Neatness & upkeep",
"ExamDate":"2017-08-03",
"MaxMark":"2.00",
"MarkObtained":"2.00" ,
"Grade":"A1"
}]
}]
}]
}]
}]
}
how can i serialize the sql result to json in mvc api,i'm already using newtonsoft for serializing other results,using LINQ is better way? if yes how?
my code Looks like
I don't know of any libraries that will do it for you automatically, but you can certainly write code that does something like:
var grouped = results.GroupBy(r => r.StudentID).Select(g => new
{
StudentID = g.Key,
ExamTypes = g.GroupBy(r => r.ExamType).Select(g2 => new
{
ExamType = g2.Key,
ExamNames = g2.GroupBy(r => r.ExamName).Select(g3 => new
{
ExamName = g3.Key,
SubjectNames = g3.GroupBy(r => r.SubjectName).Select(g4 => new
{
SubjectName = g4.Key,
SubComponents = g4.Select(r => new { SubjectComponentName = r.SubjectComponentName, ExamDate = r.ExamDate, MaxMark = r.MaxMark, MarkObtained = r.MarkObtained /* others here */ })
})
})
})
});
var serialized = Newtonsoft.Json.JsonConvert.SerializeObject(grouped);
Note that I've pluralized the names of the properties whose values are collections, but if you're required to keep to the exact property names specified, you can change that easily enough.
I have a nested elasticsearch document and I want to search within all the fields of that document i.e I want to search in both the top-level and the nested fields. My index name is people and my type name is person.
My documents look like this :
{
"id": 1,
"fname": "elizabeth",
"mname": "nicolas",
"lname": "thomas",
"houseno": "beijing",
"car": [
{
"carname": "audi",
"carno": 4444,
"color": "black"
},
{
"carname": "mercedez",
"carno": 5555,
"color": "pink"
}
]
}
Then i have the following query in .net which actually searches for an user input keyword in the elasticsearch documents. Basically, I want to search in each and every field of a document. And I use inner_hits in my query so that i can return only the matching nested document.
I have designed my query as :
var result = client.Search<person>
(s => s
.From(from)
.Size(size)
.Source(false)
.Query(query => query.Filtered(filtered => filtered
.Query(q => q.MatchAll())
.Filter(f => f.Nested(nf => nf
.InnerHits()
.Path(p => p.car)
.Query(qq => qq.Match(m => m.OnField(g => g.car.First().carname).Query(searchKeyword))))))));
And my corresponding JSON query which i use in the head plugin is :
POST-people/person/_search:
{
"_source":false,
"query": {
"filtered": {
"query": {"match_all": {}},
"filter": {
"nested": {
"path": "car",
"filter": {
"term": {
"car.carname": "searchKeyword"
}
},
"inner_hits" : {}
}
}
}
}
}
But i wanted to search in all the fields(id,fname,mname,lname,houseno,carname,carno,color) and not just in a single field e.g. in carname as i have done in my above query.
Also, i want to do partial searching like %xyz%.
How can i do these ?
Can anyone help me modify this query so that i can use this single query to search within all the fields as well as do partial searching?
I'm new to elasticsearch as well as .net,so I would be thankful for any help.
Did you try using Query instead of OnField method?
I mean, having your query this way:
var result = client.Search<person>
(s => s
.From(from)
.Size(size)
.Source(false)
.Query(query => query.Filtered(filtered => filtered
.Query(q => q.MatchAll())
.Filter(f => f.Nested(nf => nf
.InnerHits()
.Path(p => p.car)
.Query(qq => qq.Match(m => m.Query(searchKeyword))))))));