I quire data from elastic and I get a big bunch of information.
I would like to get only two properties with values (key value pairs): timestamp and value, but I get lots, all the other information too.
How can I require only my properties I want? I tried like I read at elastic.co, but I still get always to full bunch of data.
Here my tries:
var result = ElasticClient.Search<_doc>(document =>
document
.Source(sf => sf
.Includes(i => i
.Fields(
f => f.Timestamp,
f => f.Value
)
)
.Excludes(e => e
.Fields(
f => f.ContextName
)
)
)
.Query(q => q
.Match(m => m
.Field(f => f.DataRecordId)
.Query(search)
)
)
);
Or:
var result = ElasticClient.Search<_doc>(document =>
document
.StoredFields(sf => sf
.Fields(
f => f.Timestamp,
f => f.Value
)
)
.Query(q => q
.Match(m => m
.Field(f => f.DataRecordId)
.Query(search)
)
)
);
Both return a big bag of data, much more than only Timestamp and Value.
So I get still all properties, but the excluded ones are null. I hope I will reduce traffic this way for better performance.
I'm open to other solutions.
var result = ElasticClient.Search<_doc>(document =>
document
.Source(src => src
.Includes(i => i
.Fields(
p => p.Timestamp,
p => p.Value
)
)
.Excludes(e => e
.Fields(
p => p.ComponentId,
p => p.ContextName,
p => p.DataRecordId,
p => p.ResourceId
)
)
)
.Query(q => q
.Match(m => m
.Field(f => f.DataRecordId)
.Query(search)
)
)
);
Related
My Elasticsearch document looks like this; I have created an average aggregation in the buckets of 60 seconds interval
{
"version" : 4,
"metric1" : 0.688872,
"metric2" : 0.021005,
"metric3" : 0.578913,
"metric4" : 71.57523,
"metric5" : 10.71166,
"Agentid" : "12345",
"epoch" : 1638173827064
}
My Aggregation Code looks like this;
var response = await _client.SearchAsync<MyPOCO>(s => s
.Index(indexName)
.TrackTotalHits(true)
.Size(0)
.Query(q => q
.Bool(b => b
.Must(mu => mu
.Match(m => m
.Field("AgentId")
.Query(Convert.ToString(AgentId))
)
)
.Filter(fi => fi
.DateRange(r => r
.Field("epoch")
.GreaterThanOrEquals(Convert.ToString(StartTime))
.LessThan(Convert.ToString(EndTime))
.Format("epoch_millis")
)
)
)
)
.Aggregations(a => a
.DateHistogram("metricsperminute", ab => ab
.Field("ts")
.FixedInterval("1m")
.Aggregations(x => x
.Average("metric1", m => m
.Field(o => o.metric1)
)
.Average("metric2", n => n
.Field(o => o.metric2)
)
.Average("metric3", o => o
.Field(o => o.metric4)
)
.Average("metric5", p => p
.Field(o => o.metric5)
)
)))
.Sort(sort => sort.Ascending("epoch")));
I am a bit lost in how to fetch more than 10k records/buckets in the aggregations ? How to do pagination or scrolling ? Please help.
There is a search.max_buckets dynamic cluster setting though I'd try to avoid returning too many buckets at once. It's extremely slow
I'm using NEST 6.0.1 with same elasticsearch version.
I'm trying to make a select using .Terms. Alone, it works ok, but together with others filters like .Must... seems the .Terms is ignoring the .Must filters.
// Params comming in request method:
ElasticClient client, int maximumRows, string jobId, string merchantId, string category, ICollection<int> priorityFilterCollection
var searchResponse = client.Search<LogEntity>(s => s
.From(0)
.Size(maximumRows)
.Query(q => q
.Bool(b => b
.Must(
sd => sd.MatchPhrase(m => m
.Field(f => f.JobId)
.Query(jobId)
)
)
.Must(
sd => sd.MatchPhrase(m => m
.Field(f => f.MerchantId)
.Query(merchantId)
)
)
.Must(
sd => sd.MatchPhrase(m => m
.Field(f => f.Category)
.Query(category)
)
)
.Must(
sd => sd.Terms(m => m
.Field(f => f.Priority)
.Terms<int>(priorityFilterCollection)
)
)
)
)
);
A call to Must on the bool query descriptor is assignative, so with multiple calls, only the last one is assigned to the must clause. The bool query needs to be rewritten in order to pass multiple must clauses
var searchResponse = client.Search<LogEntity>(s => s
.From(0)
.Size(maximumRows)
.Query(q => q
.Bool(b => b
.Must(
sd => sd.MatchPhrase(m => m
.Field(f => f.JobId)
.Query(jobId)
),
sd => sd.MatchPhrase(m => m
.Field(f => f.MerchantId)
.Query(merchantId)
),
sd => sd.MatchPhrase(m => m
.Field(f => f.Category)
.Query(category)
),
sd => sd.Terms(m => m
.Field(f => f.Priority)
.Terms<int>(priorityFilterCollection)
)
)
)
)
);
Check out the Writing bool queries documentation
Currently working on a project involving a vehicle database.
We have set up some filters for the end users to select, such as chassis, transmission, price. Which works as intended
Now we are trying to implement a autocomplete feature in the search field. Since the simple query string will require a complete modelname and or brandname, I have looked into completion and that works for an unfinished brand name ie: "tesl" will suggest the respective Tesla models.
What I'm struggling with is getting the suggestions within the specified query.
Thanks!
Solution:
Not sure if this is the correct way to solve it, but I just ran the main query with the suggested ids to get the full payload to get all the data i need for the text search.
var result = await _client.SearchAsync<Models.VehicleModel>(descriptor => descriptor
.Index("vehiclemodel")
.Query(
q =>
q
.Terms(t => t
.Name("ModelName")
.Field("modelName")
.Terms(searchArguments.Models)
)
&&
q
.Terms(t => t
.Name("BrandName")
.Field("brandName")
.Terms(searchArguments.Brands)
)
&&
q
.Range(t => t
.Name("Price")
.Field("modelMinimumPrice")
.GreaterThanOrEquals(searchArguments.MinPrice)
.LessThanOrEquals(searchArguments.MaxPrice)
)
&&
q
.SimpleQueryString(c => c
.Name("textquery")
.Query(searchArguments.Query)
.Boost(2)
.Fields(f => f
.Field(p => p.BrandName + "^2")
.Field(p => p.ModelName + "^2")
.Field(p => p.EnergySource)
.Field("*")
)
)
public async Task<dynamic> GetModelSearchSuggestions(SearchArguments searchArguments = default)
{
var result = await _client.SearchAsync<dynamic>(s => s
.Index("vehiclemodel")
.Suggest(su => su
.Completion("searchsuggest", cs => cs
.Field("suggest")
.Prefix(searchArguments.Query)
.Fuzzy(f => f
.Fuzziness(Fuzziness.Auto)
)
)
)
);
return result.Suggest;
}
mappingDescriptor
.Properties(p => p
.Completion(cp => cp
.Name("suggest")
.Analyzer("standard")
.SearchAnalyzer("standard")
)
.Text(t => t.Name("modelName").Fielddata(true))
.Text(t => t.Name("brandName").Fielddata(true))
);
I'm pretty new to Elastic Search and stumbled upon this issue.
When searching multiple document types from the same index, the types are getting appended in the result documents set instead of being default sorted by boosted field.
My document types shares the same fields.
This is my search query:
var response = await client.SearchAsync<ProductListResponse>(s => s
.Type("product,productbundle")
.Index(index)
.From(from)
.Size(size)
.Query(fsq => fsq
.FunctionScore(c => c.Query(q => q
.MultiMatch(m => m.Query(request.Query)
.Fields(f => f
.Field(n => n.Name, 100.0)
.Field(n => n.NameWithoutSpecialChars, 100.0)
.Field(n => n.ProductName)
.Field(n => n.TeaserText)
.Field(n => n.Description)
.Field(n => n.Features)
.Field(n => n.Modules)
)
.Type(TextQueryType.PhrasePrefix)
)
).Functions(f => f
.FieldValueFactor(b => b
.Field(p => p.IsBoosted)
.Modifier(FieldValueFactorModifier.Log1P))
)
)
)
.Sort(ss => ss
.Descending(SortSpecialField.Score))
.PostFilter(filter => filter.Bool(b => b.Must(must => allFilters)))
.Source(sr => sr
.Include(fi => fi
.Field(f => f.Name)
.Field(n => n.ProductName)
.Field(n => n.TeaserText)
.Field(f => f.Image)
.Field(f => f.Thumbnail)
.Field(f => f.Url)
.Field(f => f.Features)
)
)
);
Any help is appreciated.
I would preferre not to adapt the product type with the additons to productbundle type..
I can confirm that .Type() does not mess with the order.
My issue was a boosted property not getting a value while indexing the bundles.
I'm having trouble configuring stop words in ElasticSearch using the NEST client. Here's what my index definition looks like:
var createIndexResponse = _client.CreateIndex(IndexName, c => c
.Settings(s => s
.Analysis(a => a
.Analyzers(aa => aa.Stop("pfstop", st => st.StopWords("_english_"))
)
)
)
.Mappings(m => m
.Map<SearchTopic>(mm => mm
.Properties(p => p
.Text(t => t
.Name(n => n.Posts)
.Name(n => n.FirstPost)
.Name(n => n.Title)
.SearchAnalyzer("pfstop")
)
)
)
)
);
And here's my query (and yes, I'm only wanting to return the ID):
var searchResponse = _client.Search<SearchTopic>(s => s
.Source(sf => sf.Includes(i => i.Fields(f => f.Id)))
.Query(q => q.MultiMatch(m => m.Query(searchTerm)
.Fields(f => f
.Field(x => x.Title, boost: 20)
.Field(x => x.FirstPost, boost: 2)
.Field(x => x.Posts))))
.Take(pageSize)
.Skip(startRow));
If my searchTerm is "Simon and Diana," I get results from any row that has "and" in it, which should be filtered out by way of the stop words.
Fluent syntax strikes again. After some experimentation, starting with including just one field in the mapping, I learned you have to split out the fields and their analyzer pairings. The correct syntax was this:
.Mappings(m => m
.Map<SearchTopic>(mm => mm
.Properties(p => p
.Text(t => t
.Name(n => n.Posts)
.Analyzer("pfstop")
)
.Text(t => t
.Name(n => n.FirstPost)
.Analyzer("pfstop")
)
.Text(t => t
.Name(n => n.Title)
.Analyzer("pfstop")
)
)
)
)