I'm using Microsoft SQL Server Management Studio and ElasticSearch 2.3.4 with ElasticSearch-jdbc-2.3.4.1, and i linked ES with my mssql server. Everything works fine, but when i make a query using NEST on my MVC program the result is empty. When i put an empty string inside my search attribute i get the elements, but when i try to fill it with some filter i get an empty result. Can someone help me out please? Thanks in advance.
C#:
const string ESServer = "http://localhost:9200";
ConnectionSettings settings = new ConnectionSettings(new Uri(ESServer));
settings.DefaultIndex("tiky");
settings.MapDefaultTypeNames(map => map.Add(typeof(DAL.Faq), "faq"));
ElasticClient client = new ElasticClient(settings);
var response = client.Search<DAL.Faq>(s => s.Query(q => q.Term(x => x.Question, search)));
var result = response.Documents.ToList();
DAL:
Postman:
PS: i followed this guide to create it
EDIT:
Index mapping:
There's a couple of things that I can see that may help here:
By default, NEST camel cases POCO property names when serializing them as part of the query JSON in the request, so x => x.Question will serialize to "question". Looking at your mapping however, field names in Elasticsearch are Pascal cased, so what the client is doing will not match what's in Elasticsearch.
You can change how NEST serializes POCO property names by using .DefaultFieldNameInferrer(Func<string, string>) on ConnectionSettings
const string ESServer = "http://localhost:9200";
ConnectionSettings settings = new ConnectionSettings(new Uri(ESServer))
.DefaultIndex("tiky");
.MapDefaultTypeNames(map => map.Add(typeof(DAL.Faq), "faq"))
// pass POCO property names through verbatim
.DefaultFieldNameInferrer(s => s);
ElasticClient client = new ElasticClient(settings);
As Rob mentioned in the comments, a term query does not analyze the query input. When executing a term query against a field that is analyzed at index time then, in order to get matches, the query text that you pass to the term query would need to take the analysis that is applied at index time into account. For example,
Question is analyzed with the Standard Analyzer
A Question value of "What's the Question?" will be analyzed and indexed as tokens "what's", "the" and "question"
A term query would need to have a query input of "what's", "the" or "question" to be a match
A match query, unlike a term query, does analyze the query input, so the output of the search analysis will be used to find matches. In conjunction with Pascal casing highlighted in 1., you should now get documents returned.
You can also have the best of both worlds in Elasticsearch i.e. analyze input at index time for full-text search functionality as well as index input without analysis to get exact matches. This is done with multi-fields and here is an example of creating a mapping that indexes Question properties as both analyzed and not analyzed
public class Faq
{
public string Question { get; set; }
}
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var defaultIndex = "default-index";
var connectionSettings = new ConnectionSettings(pool)
.DefaultIndex(defaultIndex)
.DefaultFieldNameInferrer(s => s);
var client = new ElasticClient(connectionSettings);
if (client.IndexExists(defaultIndex).Exists)
client.DeleteIndex(defaultIndex);
client.CreateIndex(defaultIndex, c => c
.Mappings(m => m
.Map<Faq>(mm => mm
// let NEST infer mapping from the POCO
.AutoMap()
// override any inferred mappings explicitly
.Properties(p => p
.String(s => s
.Name(n => n.Question)
.Fields(f => f
.String(ss => ss
.Name("raw")
.NotAnalyzed()
)
)
)
)
)
)
);
The mapping for this looks like
{
"mappings": {
"faq": {
"properties": {
"Question": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
The "raw" sub field under the "Question" field will index the value of the Question property without any analysis i.e. verbatim. This sub field can now be used in a term query to find exact matches
client.Search<Faq>(s => s
.Query(q => q
.Term(f => f.Question.Suffix("raw"), "What's the Question?")
)
);
which find matches for the previous example.
Related
Is there a way to replace the value of one result with another after doing a linq statement?
is that in my linq statement in ** Status **, there are some that come to me with this result "PendingForApprover" but I want to replace it with "Pending for approver", I would like that after obtaining the result, you can use a find and based on that replace the values, before in my model I had it with a data annotation with a display name but for the reason that that result I am going to transform it into JSON and then it will be brought by server side that method is not working
y share my sentence LINQ
var result = db.document.Select(d => new DocumentViewModel
{
DocumentId = d.DocumentId,
Name = w.name
ReceivedLogs = d.Logs
Status = w.Status.toString(),
.Where(l => l.Status == Status.Received)
.Select(l => new LogViewModel
{
CurrentApprover = l.User,
NameApprover = l.User.FullName
}).FirstOrDefault()
}).ToList();
thanks
Have you tried with Switch expressions it is a pretty clear solution and you can integrate it in your LINQ query.
Status = w.Status switch
{
Status.StatusOne => "Status One",
Status.PendingForApprover => "Pending for approver",
_ => "Unknown"
}
Using EF Core 2.1 which does support Group By translation but not after I project the key value.
I have a query that needs to allow for a range of grouping types and a range of aggregate types.
Group By: Year, Year/Month, Year/Month/Day etc
Aggregate By: Avg, Sum, Min, Max etc.
I created two switch statements, one for the grouping and one for the aggregation. My issue is that I am not able to defer execution and perform the grouping within SQL. If not, the resulting data set is quite large.
Is any of this a reasonable approach or should I just use a raw query?
This is deferred and grouped as desired but the key type is unique to year/month and I am not able to provide a general solution for the aggregates in the second switch if I wanted to group by something different such as year/month/day.
var query1 = series
.GroupBy(k => new { k.Created.Year, k.Created.Month, });
var result1 = query1
.Select(i => new { i.Key, Value = i.Average(a => a.DataValue) });
As soon as you convert the key to a string, grouping occurs client-side:
var query2 = series
.GroupBy(k => k.Created.Year.ToString());
var result2 = query2
.Select(i => new { i.Key, Value = i.Average(a => a.DataValue) });
This too causes grouping to occur client side:
var query3 = series
.GroupBy(k => new { k.Created.Year, k.Created.Month, })
.Select(i => new
{
Key = $"{i.Key.Year}-{i.Key.Month}",
Values = i.ToList()
});
Any ideas how to accomplish my query? Ideally I need a common group key that groups server-side or a method of passing the aggregate based on a function in the query. Generating a string based key seems to ensure that the grouping occurs client side.
The only way I was able to get SQL translation in EF Core 2.1 (and 2.2) was to group by composite anonymous type conditionally including the desired data parts:
bool includeYear = ...;
bool includeMonth = ...;
bool includeDay = ...;
bool includeHour = ...;
var query1 = series
.GroupBy(e => new
{
Year = includeYear ? e.CreatedDate.Year : 0,
Month = includeMonth ? e.CreatedDate.Month : 0,
Day = includeDay ? e.CreatedDate.Day : 0,
Hour = includeHour ? e.CreatedDate.Hour : 0
});
Weirdly enough, if I create a special class for group key with exactly the same properties as the anonymous type and change new { ... } to new MyKeyType { ... }, EF Core switches to client evaluation.
Shortly, GroupBy (and not only) SQL translation is still unstable. Looks like we have to wait for 3.0 to get eventually improvements in that area.
Using the C# NEST API on Elasticsearch:
var searchResults = client.Search<Product>(s => s
.Index(Constants.ElasticSearchIndex)
.Query(q => q
.Raw(jsonRequest)
)
);
The query is supposed to run on the /sc_all/ index, but it runs on /sc_all/product/ index (which doesn't exist - /product/ seems to be added because of the Search since T = product).
If I do like this, /product/ is replaced with the value of the constant, i.e. /sc_all/product/ => /sc_all/constant_value/:
var searchResults = client.Search<Product>(s => s
.Index(Constants.ElasticSearchIndex)
.Type(Constants.ElasticSearchType)
.Query(q => q
.Raw(jsonRequest)
)
);
What should I do if I just want to query /sc_all/ and nothing else?
Thanks!
Json request:
"{\"filtered\": {\"query\": {\"match_all\": { }},\"filter\": {\"nested\" : {\"path\" : \"products\",\"filter\": {\"nested\" : {\"path\" : \"products.da\",\"filter\": { \"bool\": { \"must\": [{\"query\": {\"query_string\": {\"default_field\" : \"products.da.content\", \"query\" : \"kildemoes\"}}}]}}}}}}}}, \"from\": 0, \"size\": 100"
You just need to specify to run across all types with .AllTypes()
var jsonRequest = "{ \"match_all\": {} }";
var searchResults = client.Search<Product>(s => s
.Index(Constants.ElasticSearchIndex)
.AllTypes()
.Query(q => q
.Raw(jsonRequest)
)
);
which will generate the following request
POST http://localhost:9200/sc_all/_search
{
"query": { "match_all": {} }
}
Bear in mind that any documents returned will attempt to be deserialized into instances of Product so if you will be targeting multiple different types, you may want to use a common base type or dynamic and additionally, take advantage of covariant search results.
You were using an outdated version client, like 5.x.
I came across the same problem too using 5.x. The second subpath is the document type, which is the _type name of your document and is docs by default. So, the solution I use is to add [ElasticsearchType(Name = "docs")] at the top of my POCO class and the search path will is something like /{indexname}/docs/_search, and it's fine.
public class User
{
public string Email { get; set; }
}
client.Index(new User { Email ="test#test.te" });
Query in Linq C# for example :
rep.Where(user=>user.Email=="test#test.te");
That works correctly.
I use same query in Nest:
client.Search<Post>(q => q
.Query(qu => qu
.Term(te=>te.OnField("email").Value("test#test.te"))));
Document result count is zero!!
But :
client.Search<Post>(q => q
.Query(qu => qu
.Term(te=>te.OnField("email").Value("test"))));
Document result count is 1 - why?
How I can make equal-query in ElasticSearch?
It's all because analyzers. Your document is parsed to terms while indexing, which means that elasticsearch stores something like an array of strings ["test", "test", "te"] in row with your document. Depending on what anayzer is configured (I guess it's standard one), you may get different terms decomposition. On the other side, your term request is not analyzed; that's why first request returns nothing - there's no such string as "test#test.te" in index strings ["test", "test", "te"], but there's a "test" string, so you get results for the second one. In your case you should use query_string query, but beware of the fact that such queries are analyzed too. It means, that if you index two documents, say {"Email":"test#test.te"} and {"Email":"test#gmail.com"}, without any flags query {"query":{"query_string":{"default_field":"Email","query":"test#test.te"}}} will return both documents because both of them contain "test" string in the index. To avoid this, use something like {"default_field":"Email","query":"test#test.te", "default_operator":"AND"}}} - default default operator is "OR".
Speaking of NEST, use query like
client.Search<Post>(q => q
.Query(qu => qu
.QueryString(qs=>qs
.OnField(x=>x.Email).Query("test#test.te").Operator(Operator.and)
)
)
);
I'm a complete beginner to elasticsearch and I have been trying to use elasticsearch's completion suggester using Nest for auto-complete on a property.
Here is my mapping (as mentioned here: ):
var createResult = client.CreateIndex(indexName, index => index
.AddMapping<Contact>(tmd => tmd
.Properties(props => props
.Completion(s =>
s.Name(p => p.CompanyName.Suffix("completion"))
.IndexAnalyzer("standard")
.SearchAnalyzer("standard")
.MaxInputLength(20)
.Payloads()
.PreservePositionIncrements()
.PreserveSeparators())
)
)
);
var resultPerson = client.IndexMany(documents.OfType<Person>(), new SimpleBulkParameters { Refresh = true });
var resultCompany = client.IndexMany(documents.OfType<Company>(), new SimpleBulkParameters { Refresh = true });
And while indexing I'm just making use of IndexMany method and passing the IEnumberable<Contact> (Contact has a property by name CompanyName, Contact is an abstract class, both Person and Company are concrete implementations of it). The search throws an exception saying ElasticSearchException[Field [companyName] is not a completion suggest field]. And the query looks like below:
SearchDescriptor<Contact> descriptor = new SearchDescriptor<Contact>();
descriptor = descriptor.SuggestCompletion("suggest", c => c.OnField(f => f.CompanyName).Text(q));
var result = getElasticClientInstance("contacts").Search<Contact>(body => descriptor);
string qe = result.ConnectionStatus.ToString();
What am I doing wrong here, I looked into Nest's tests on SuggestCompletion but not much help, meaning the test only depict on how to get suggestions but not on how to set index mappings for SuggestCompletion.
I also tried setting up edgeNgram tokenizer as mentioned in this post but, couldn't proceed there as well.
Any direction or an example on how to proceed would greatly help.
UPDATE
You are try to create a property with the name "companyName.completion" but at that position its not valid and it will use the last token "completion". So its actually mapping a field called completion.... try changing the call to: .Name(p => p.CompanyName)
Other observations
You specify a mapping for the Contact but while indexing you use the Person and Company types.
In elasticsearch terms you mapped:
/index/contact/
but your documents are going into:
/index/person/ and /index/company
NEST won't automatically map all implementation of a specific class and elasticsearch has no way of knowing the three are related.
I would refactor the mapping to a method and call it for all the types involved.
var createResult = client.CreateIndex(indexName, index => index
.AddMapping<Contact>(tmd => MapContactCompletionFields(tmd))
.AddMapping<Person>(tmd => MapContactCompletionFields(tmd))
.AddMapping<Company>(tmd => MapContactCompletionFields(tmd))
);
private RootObjectMappingDescriptor<TContact> MapContactCompletionFields<TContact>(
RootObjectMappingDescriptor<TContact> tmd)
where TContact : Contact
{
return tmd.Properties(props => props
.Completion(s => s
.Name(p => p.CompanyName.Suffix("completion"))
.IndexAnalyzer("standard")
.SearchAnalyzer("standard")
.MaxInputLength(20)
.Payloads()
.PreservePositionIncrements()
.PreserveSeparators()
)
);
}
That method returns the descriptor so you can further chain on it.
Then when you do a search for contacts:
var result = getElasticClientInstance("contacts").Search<Contact>(
body => descriptor
.Types(typeof(Person), typeof(Company))
);
That types hint will cause the search to looking /index/person and /index/company and will know how to give you back a covariant list of documents.
So you can do result.Documents.OfType<Person>() after the previous call.