I have a problem with doing grouping operation in elastcicSearch.
Actually, I have 3 fields in my document. that is as under:
Id Type Year
Now I want to do grouping on ExceptionType and Year and count it in "ResultCount".
I tried this one but it is not working:
.Aggregations(a => a
.ValueCount("ResultCount", c => c
.Field(p => p.Id)
.Field(p=> p.Year)
))
.Aggregations(a => a
.Terms("Type", st => st
.Field(o => o.Type)
.Size(10))).Size(5)
.Aggregations(aa => aa
.Max("Year", m => m
.Field(o => o.Year)
))
);
Please give a solution for this problem as soon as possible. thank you.
Here I will try to help. There is no such thing as group in elastic only terms and sub aggregations. If you want the count of error types by year you can do it like so:
var response = db.Search<Error>(s =>
s.Size(0)
.Aggregations(aggs1 => aggs1.Terms("level1",
l1 =>l1.Field(f => f.Type)
.Aggregations(aggs2 => aggs2.Terms("leve2", t=> t.Field(f=>f.Year))))));
foreach (var l1 in response.Aggs.Terms("level1").Buckets)
{
foreach (var l2 in l1.Terms("leve2").Buckets)
{
Console.WriteLine("Type:{0}, Year:{1}, Count:{2}", l1.Key, l2.Key, l2.DocCount);
}
}
But keep in mind that this will only work as you except for a non-analysed or keyword fields if you want terms for dates you can use something like date_histogram aggregation.
If you index mapping looks like this
{
"properties": {
"type": {
"type": "keyword"
},
"year": {
"type": "long"
}
}
}
You will get the number of items per error type per year
You can do other types of aggregations as well in the nested aggregation
NOTE: The terms aggregation is approximated so don't be surprised if you don't get exact values.
Related
I am using this hopelessly inefficient code to establish if a document is already indexed:
foreach (var entry in dic)
{
var response = client.Search<Document>(s => s.Query(q => q.QueryString(d =>
d.Query(string.Format("{0}", entry.Key)))));
if (response.Documents.Count == 0)
{
not_found++;
}
else
{
found++;
}
}
I wonder, if one could send several entry.Key in one batch rather than hitting the endpoint for every id (entry.Key)? Thanks.
Sure!
You can use a terms filter:
client.Search<Document>(s => s.Query(
q => q.Terms(
c => c
.Field(doc => doc.Id)
.Terms(keys)))
If you are specifically looking for IDs, you can use the ids filter:
client.Search<Document>(s => s.Query(
q => q.Ids(c => c.Values(keys))
);
If you are only interested in whether or not the document(s) have been indexed, consider limiting the returned fields to only the ID field so you don't waste bandwidth returning the full document:
response = client.Search<Document>(s => s
.Query(q => q.Ids(c => c.Values(keys)) // look for these IDs
.StoredFields(sf => sf.Fields(doc => doc.Id)) // return only the Id field
);
Lastly, if you're only interested in the number of matching documents, then you can ask Elasticsearch to not return any results, and only use the response metadata to count how many documents matched:
response = client.Search<Document>(s => s
.Query(q => q.Ids(c => c.Values(keys))) // look for these IDs
.Size(0) // return 0 hits
);
found += response.Total; // number of total hits
I'm having a tough time finding information on how to search nested properties using the Nest client in C#.
I have email objects in an index with approximately this shape:
{
subject: “This is a test”,
content: “This is an email written as a test of the Elasticsearch system. Thanks, Mr Tom Jones”,
custodians: [
{
firstName: “Tom”,
lastName: “Jones”,
routeType: 0
},
{
firstName: “Matthew”,
lastName: “Billsley”,
routeType: 1
}
]
}
You should be able to see that there is an array in there called “custodians” which is a list of all the senders and recipients of the email. In the Fluent-style query builder in .Net I can build the query just fine when I’m using subject, content, and other “first tier” properties. But I may only want to include custodians who have the routeType = 0 in some queries. I can’t seem to find any guidance on how to accomplish this. Any ideas?
For instance, a query for the term “picnic” in the subject field would look like:
Client.SearchAsync(m => m
.Query(q => q
.Match(f => f
.Field(msg => msg.Subject)
.Query(“picnic”))));
What would the query to only get messages from the index with routeType = 0 and lastName = “Jones” be?
FYI: This is crossposted to the Elasticsearch forums. If I get a good suggestion there, I will add it here.
If you want to get messages that have a custodian with routeType == 0:
Client.SearchAsync(m => m
.Query(q => q
.Term(t => t
.Field(msg => msg.Custodians[0].RouteType)
.Value(0))));
If you want to get messages that have a custodian with lastName == "jones":
Client.SearchAsync(m => m
.Query(q => q
.Term(t => t
.Field(msg => msg.Custodians[0].LastName)
.Value("jones"))));
If you want to get messages that have a custodian with lastName == "jones" AND routeType == 0:
Client.SearchAsync(m => m
.Query(q => q
.Nested(t => t
.Path(msg => msg.Custodians)
.Query(nq =>
nq.Term(t => t.Field(msg => msg.Custodians[0].RouteType).Value(0) &&
ng.Term(t => t.Field(msg => msg.Custodians[0].LastName).Value("jones")
)
)
)
);
Note that custodians will need to be mapped as a nested field for the last query to work as expected. See here for more about nested fields.
situation is this. I have in elastic a group. each of these groups have a nested list of items.
Both group and items have an attribute named serial, which are unique.
I get a serial for the group and a serial for item, and with those 2 items i'm supposed to return the item.
Currently i'm doing it the following way:
public item findItem(string groupSerial, string itemSerial)
{
var searchResponse = _elasticClient.Search<Group>(s => s
.Index(_config.groupIndexName)
.Query(q => q
.ConstantScore(cs => cs
.Filter(f => f
.Term(t => t
.Field(fi => fi.serial)
.Value(groupSerial)
)
)
)
).Query(q => q
.Nested(c => c
.InnerHits(i => i.Explain())
.Path(p => p.items)
.Query(nq => nq.Term(t => t
.Field(field => field.items.First().serial)
.Value(itemSerial)))))
);
var result = searchResponse.Documents.FirstOrDefault();
return result?.items.Find(item => item.serial == itemSerial);
}
I get the feeling that there is supposed to be a more efficient way. Like getting the item straight from the search in elastic. Does anyone know how?
Hi have a query in which i have to match using fuzzy operation.
{
"query": {
"match": {
"answer": {
"query": "conevrt o",
"fuzziness": 2
}
}
}
}
when i try to write this using Lambda expressions style im getting an error saying cannot convert int to NEST.fuzziness.
here is my take on lambda expression.
match = drFuzzy.Text; //im getting text from the asp:dropdown(hardcoded 0,0.5,1,2)
int fuzz = Int32.Parse(match); // converting this to integer
var searchResponse = client.Search<StudResponse>(s => s
.Query(q => q
.Match(m => m
.Field(f => f.Answer)
.Query(key1)
.Fuzziness(fuzz) //throwing an error here. cannot convert from int to Nest.Fuzziness
)
)
);
Thanks in advance.
To pass fuzinness parameter you will have to use Fuzziness class and EditDistance method. NEST documentation has a really nice example of match query usage, have a look.
This is how you can use Fuzziness.EditDistance(..) code in your use case.
client.Search<StudResponse>(s => s
.Query(q => q
.Match(m => m
.Field(f => f.Answer)
.Query(key1)
.Fuzziness(Fuzziness.EditDistance(fuzz))
)
));
Hope it helps.
I need to query certain fields, after aggregating.
Document structure is:
{
id: 1,
type: AA,
hashValue: "qweqeqwdwwew"
...and many more fields
}
I want to aggregate by 'hashValue', so that i get only unique hashValues and the return results should also have the type. I need help with NEST query.
The current query to aggregate is:
var result = esClient.Search < EType > (q => q
.Index(esClient.Index)
.Routing(id.ToString(CultureInfo.InvariantCulture))
.Aggregations(ag => ag
.Terms("Hash", ee => ee
.Field(f => f.hashValue)))));
How can i extend it return type field along with hashValue?
Thanks.
From your comments, it seems like you want to aggregate documents per type per hash value. For that the Nest query you need is as under:
var result = esClient.Search<EType>(q => q
.Index(esClient.Index)
.Routing(id.ToString(CultureInfo.InvariantCulture)
.Aggregations(agHash => agHash
.Terms("Hash", eeHash => eeHash
.Field(fHash => fHash.hashValue)
.Aggregations(agType => agType
.Terms("Types", eeType => eeType
.Field(fType => fType.typeValue))))));