I'm having a tough time finding information on how to search nested properties using the Nest client in C#.
I have email objects in an index with approximately this shape:
{
subject: “This is a test”,
content: “This is an email written as a test of the Elasticsearch system. Thanks, Mr Tom Jones”,
custodians: [
{
firstName: “Tom”,
lastName: “Jones”,
routeType: 0
},
{
firstName: “Matthew”,
lastName: “Billsley”,
routeType: 1
}
]
}
You should be able to see that there is an array in there called “custodians” which is a list of all the senders and recipients of the email. In the Fluent-style query builder in .Net I can build the query just fine when I’m using subject, content, and other “first tier” properties. But I may only want to include custodians who have the routeType = 0 in some queries. I can’t seem to find any guidance on how to accomplish this. Any ideas?
For instance, a query for the term “picnic” in the subject field would look like:
Client.SearchAsync(m => m
.Query(q => q
.Match(f => f
.Field(msg => msg.Subject)
.Query(“picnic”))));
What would the query to only get messages from the index with routeType = 0 and lastName = “Jones” be?
FYI: This is crossposted to the Elasticsearch forums. If I get a good suggestion there, I will add it here.
If you want to get messages that have a custodian with routeType == 0:
Client.SearchAsync(m => m
.Query(q => q
.Term(t => t
.Field(msg => msg.Custodians[0].RouteType)
.Value(0))));
If you want to get messages that have a custodian with lastName == "jones":
Client.SearchAsync(m => m
.Query(q => q
.Term(t => t
.Field(msg => msg.Custodians[0].LastName)
.Value("jones"))));
If you want to get messages that have a custodian with lastName == "jones" AND routeType == 0:
Client.SearchAsync(m => m
.Query(q => q
.Nested(t => t
.Path(msg => msg.Custodians)
.Query(nq =>
nq.Term(t => t.Field(msg => msg.Custodians[0].RouteType).Value(0) &&
ng.Term(t => t.Field(msg => msg.Custodians[0].LastName).Value("jones")
)
)
)
);
Note that custodians will need to be mapped as a nested field for the last query to work as expected. See here for more about nested fields.
Related
I am using this hopelessly inefficient code to establish if a document is already indexed:
foreach (var entry in dic)
{
var response = client.Search<Document>(s => s.Query(q => q.QueryString(d =>
d.Query(string.Format("{0}", entry.Key)))));
if (response.Documents.Count == 0)
{
not_found++;
}
else
{
found++;
}
}
I wonder, if one could send several entry.Key in one batch rather than hitting the endpoint for every id (entry.Key)? Thanks.
Sure!
You can use a terms filter:
client.Search<Document>(s => s.Query(
q => q.Terms(
c => c
.Field(doc => doc.Id)
.Terms(keys)))
If you are specifically looking for IDs, you can use the ids filter:
client.Search<Document>(s => s.Query(
q => q.Ids(c => c.Values(keys))
);
If you are only interested in whether or not the document(s) have been indexed, consider limiting the returned fields to only the ID field so you don't waste bandwidth returning the full document:
response = client.Search<Document>(s => s
.Query(q => q.Ids(c => c.Values(keys)) // look for these IDs
.StoredFields(sf => sf.Fields(doc => doc.Id)) // return only the Id field
);
Lastly, if you're only interested in the number of matching documents, then you can ask Elasticsearch to not return any results, and only use the response metadata to count how many documents matched:
response = client.Search<Document>(s => s
.Query(q => q.Ids(c => c.Values(keys))) // look for these IDs
.Size(0) // return 0 hits
);
found += response.Total; // number of total hits
I have a database request which looks as follows:
var userHasProfessions = xy.GetUserProfessions();
var users = sessionService.GetDefaultSession()
.Query<User>()
.Where(a => userProfessions.Contains(a.Profession.Id))
.ToList();
It gets all users that the requesting user is allowed to see, depending on his own profession. Now I wanna restrict that a little more, that he can only see some attributes of the users. Let's say for example:
id -> yes
firstname -> yes
lastname -> yes
address -> no!
Now I tried t change the query to something like:
var userHasProfessions = xy.GetUserProfessions();
var users = sessionService.GetDefaultSession()
.QueryOver<User>()
.SelectList(list => list
.Select(a => a.id)
.Select(a => a.firstname)
.Select(a => a.lastname))
.Where(a => userProfessions.Contains(a.Profession.Id))
.ToList();
Now my question... Is there a way to, for example, make a new List with these attributes and then loop through it? Something like that:
List<string> attributes = new List<string>(){"id", "firstname", "lastname"}
var userHasProfessions = xy.GetUserProfessions();
var users = sessionService.GetDefaultSession()
.QueryOver<User>()
.SelectList(
//loop through attributes
)
.Where(a => userProfessions.Contains(a.Profession.Id))
.ToList();
Thanks in advance :-)
EDIT
To make my question a little bit more clear. I wanna have the attributes that the user is allowed to see, dynmically changeable from a List<string> outside the query.
How can a achieve that?
You could do
.Where(a => userProfessions.Contains(a.Profession.Id))
.Select(c => new User {
id = c.id,
firstname = c.firstname,
lastname = c.lastname,
address c.address
}).ToList();
I am assuming list is type of User. Other wise you could use anonymous return i.d. c => new { id="id",... }
situation is this. I have in elastic a group. each of these groups have a nested list of items.
Both group and items have an attribute named serial, which are unique.
I get a serial for the group and a serial for item, and with those 2 items i'm supposed to return the item.
Currently i'm doing it the following way:
public item findItem(string groupSerial, string itemSerial)
{
var searchResponse = _elasticClient.Search<Group>(s => s
.Index(_config.groupIndexName)
.Query(q => q
.ConstantScore(cs => cs
.Filter(f => f
.Term(t => t
.Field(fi => fi.serial)
.Value(groupSerial)
)
)
)
).Query(q => q
.Nested(c => c
.InnerHits(i => i.Explain())
.Path(p => p.items)
.Query(nq => nq.Term(t => t
.Field(field => field.items.First().serial)
.Value(itemSerial)))))
);
var result = searchResponse.Documents.FirstOrDefault();
return result?.items.Find(item => item.serial == itemSerial);
}
I get the feeling that there is supposed to be a more efficient way. Like getting the item straight from the search in elastic. Does anyone know how?
I have a problem with doing grouping operation in elastcicSearch.
Actually, I have 3 fields in my document. that is as under:
Id Type Year
Now I want to do grouping on ExceptionType and Year and count it in "ResultCount".
I tried this one but it is not working:
.Aggregations(a => a
.ValueCount("ResultCount", c => c
.Field(p => p.Id)
.Field(p=> p.Year)
))
.Aggregations(a => a
.Terms("Type", st => st
.Field(o => o.Type)
.Size(10))).Size(5)
.Aggregations(aa => aa
.Max("Year", m => m
.Field(o => o.Year)
))
);
Please give a solution for this problem as soon as possible. thank you.
Here I will try to help. There is no such thing as group in elastic only terms and sub aggregations. If you want the count of error types by year you can do it like so:
var response = db.Search<Error>(s =>
s.Size(0)
.Aggregations(aggs1 => aggs1.Terms("level1",
l1 =>l1.Field(f => f.Type)
.Aggregations(aggs2 => aggs2.Terms("leve2", t=> t.Field(f=>f.Year))))));
foreach (var l1 in response.Aggs.Terms("level1").Buckets)
{
foreach (var l2 in l1.Terms("leve2").Buckets)
{
Console.WriteLine("Type:{0}, Year:{1}, Count:{2}", l1.Key, l2.Key, l2.DocCount);
}
}
But keep in mind that this will only work as you except for a non-analysed or keyword fields if you want terms for dates you can use something like date_histogram aggregation.
If you index mapping looks like this
{
"properties": {
"type": {
"type": "keyword"
},
"year": {
"type": "long"
}
}
}
You will get the number of items per error type per year
You can do other types of aggregations as well in the nested aggregation
NOTE: The terms aggregation is approximated so don't be surprised if you don't get exact values.
I have an sql table with columns Name, Category, Location. I am using Elastic Search with NEST. my query like this:
var result = client.Search<Models.Search.Poll>(s => s.Query(q => q.Fuzzy(f => f.OnField(p => p.Name).Value(query))))));
So if there is a record with name = "We are here" and user search "are" , it returns result.
Now I need to add two more parameters category and location to this query:
so I made it like this:
var result = client.Search<Models.Search.Poll>(s => s.Query(q => q.Fuzzy(f => f.OnField(p => p.Name).Value(query).OnField(r => r.Category).Value(category))));
but it is not working with query field now. but it works with category now. here is what I get when I type name but dont select category:
StatusCode: OK,
Method: POST,
Url: http://server.abc.com:9200/pollit-dev/polls/_search,
Request: {
"query": {
"fuzzy": {
"category": {
"value": "Select a Category"
}
}
}
},
Response: {"took":2892,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
I tried this one well:
var result = client.Search<Models.Search.Poll>(s => s.MatchAll().Query(q => q.Term(p => p.Name, query) || q.Term(p => p.Category,category) || q.Term(p => p.Location, Location)
but no luck so far.
Regards,
Asif Hameed
You have multiple options for that.
First one is almost like yours but you have to use a Bool condition there.
var result = client.Search<Models.Search.Poll>(s => s
.MatchAll()
.Query(q => q
.Bool(b => b
.Must(m => m.Term(p => p.Name, query) || m.Term(p => p.Category,category) || m.Term(p => p.Location, Location))
)
)
);
Now another option is to use queryContainers. Something like this:
var result = _Instance.Search<Models.Search.Poll>(q => q
.Query(qq =>
{
QueryContainer termQuery = null;
QueryContainer locationQuery = null;
QueryContainer categoryQuery = null;
termQuery = qq.Term(p => p.Name, query);
categoryQuery = qq.Term(p => p.Category,category);
locationQuery = qq.Term(p => p.Location, Location);
return termQuery || categoryQuery || locationQuery;
})
);
You can also elaborate the QueryContainers and add multiple search parameters there.
Hope this helps you. Great day!