Develop Reference

Why escaping backslashes are appear in ElasticSearch Nest Query?

I am trying to write a C# method that get the queryString of a controller and converts it into an ElasticSearch query, like below:
public QueryContainerDescriptor<T> Convert<T> (IQueryCollection query) where T: class
{
var selector = new QueryContainerDescriptor<T>();
List<QueryContainer> Must = new List<QueryContainer>();
foreach(var key in query.keys)
{
string value = query[key];
var match = new MatchQuery { Field = $"{key}.keyword", Query = value };
list.Add(match)
}
selector.Bool(q => q.Must(Must.ToArray()));
return selector;
}
It works as expected, but if I pass a queryString value with a backslash, like:
http://localhost:5000/api/indexData?user=ESKA\\USER
It should be converted into this query:
{ "bool": { "must": [ { "match" : { "user.keyword": "ESKA\\USER" } } ] }
But ElasticSearch will return nothing because the value will of the query be: ESKA\\\\USER with 4 backslashes, like:
{ "bool": { "must": [ { "match" : { "user.keyword": "ESKA\\\\USER" } } ] }
how can I solve this issue?

I don't think that Nest is performing any escaping of backslashes. Here's an example that writes out the requests (and responses, if using an IConnection that sends the request)
private static void Main()
{
var pool = new SingleNodeConnectionPool(new Uri($"http://localhost:9200"));
var settings = new ConnectionSettings(pool, new InMemoryConnection())
.DefaultIndex("default_index")
.DisableDirectStreaming()
.PrettyJson()
.OnRequestCompleted(callDetails =>
{
if (callDetails.RequestBodyInBytes != null)
{
var json = JObject.Parse(Encoding.UTF8.GetString(callDetails.RequestBodyInBytes));
Console.WriteLine(
$"{callDetails.HttpMethod} {callDetails.Uri} \n" +
$"{json.ToString(Newtonsoft.Json.Formatting.Indented)}");
}
else
{
Console.WriteLine($"{callDetails.HttpMethod} {callDetails.Uri}");
}
Console.WriteLine();
if (callDetails.ResponseBodyInBytes != null)
{
Console.WriteLine($"Status: {callDetails.HttpStatusCode}\n" +
$"{Encoding.UTF8.GetString(callDetails.ResponseBodyInBytes)}\n" +
$"{new string('-', 30)}\n");
}
else
{
Console.WriteLine($"Status: {callDetails.HttpStatusCode}\n" +
$"{new string('-', 30)}\n");
}
});
var client = new ElasticClient(settings);
var collection = new QueryCollection(new Dictionary<string, StringValues>
{
{ "user", "ESKA\\USER" }
});
var response = client.Search<object>(s => s
.Query(q => Convert<object>(q, collection))
);
}
public static QueryContainerDescriptor<T> Convert<T>(QueryContainerDescriptor<T> selector, IQueryCollection query) where T : class
{
List<QueryContainer> Must = new List<QueryContainer>();
foreach (var key in query.Keys)
{
string value = query[key];
var match = new MatchQuery { Field = $"{key}.keyword", Query = value };
Must.Add(match);
}
selector.Bool(q => q.Must(Must.ToArray()));
return selector;
}
the resulting query is
POST http://localhost:9200/default_index/_search?pretty=true&typed_keys=true
{
"query": {
"bool": {
"must": [
{
"match": {
"user.keyword": {
"query": "ESKA\\USER"
}
}
}
]
}
}
}
If the user value were to be the verbatim string literal #"ESKA\\USER", then the resulting query would be
"user.keyword": {
"query": "ESKA\\\\USER"
}
because each \ in the verbatim string literal needs to be escaped.

How can I select the value and type of a dynamic list?

I have a dynamic query that brings me dynamic columns (System.Linq.Dynamic.Core):
var data = data.Select("new (col1, col2, col3)", "T", StringComparison.OrdinalIgnoreCase);
[
{ col1: "This", col2: 1.56, col3: "Something else" }
]
Now I need to bring the type of each column. ex:
[
{
col1: { value: "This", type: "string" },
col2: { value: 1.56, type: "decimal" }
}
]
To get that, I was thinking of something like:
var rows = data.Select(x => new {
Type = MyHelper.GetType(x),
Value = x
});
But that it's not working.
An unhandled error occurred The best overloaded method match for
'MyHelper.GetType(System.Reflection.PropertyInfo)' has some invalid
arguments
I do know how to get the type
public static string GetType(PropertyInfo type)
{
return type.PropertyType.IsGenericType && type.PropertyType.GetGenericTypeDefinition() == typeof(Nullable<>) ? type.PropertyType.GetGenericArguments()[0].ToString().Replace("System.", "") : type.PropertyType.Name;
}
But I don't know how to build the query.
[UPDATE]
This is a try, but not what I need
var rows = data.Select(x => new {
Type = MyHelper.GetType(x.GetType()),
Value = x
});
[
{
"type": "<>f__AnonymousType0`3",
"value": {
"col1": "2019-10-15T00:00:00",
"col2": 0.00,
"col3": "abc"
}
},
]

Here is a working demo you could refer to:
public List<object> Get(int id)
{
var data = new List<Object>
{
new{
col1="aaa",
col2= 1.25,
col3 = 1
},
new{
col2="asd",
col3= 1.2,
col4 =2
}
};
var aaa = new List<object>() { };
foreach (var item in data)
{
var ds = item.GetType().GetProperties();
foreach(var i in ds)
{
var name = i.Name;
var type = i.PropertyType.Name;
var value = item.GetType().GetProperty(name).GetValue(item);
aaa.Add(new { name=name, type = type, value = value });
}
}
return aaa;
}

How to serialize sql data without the column names?

I am Serializing data from SQL database to JSON, how can I serialize just the values without the string name OR a function to trim the serialized JSON before Deserializing.
I read about ScriptIgnoreAttribute but didn't see how to relate it with what I want to do
Original JSON
​[
{
"CODE": "AF",
"TOTALVALUE": "$23,554,857.27"
},
{
"CODE": "AS",
"TOTALVALUE": "$38,379,964.65"
},
{
"CODE": "SG",
"TOTALVALUE": "$24,134,283.47"
}
]
Desired JSON
​[
{
"AF": "$23,554,857.27"
},
{
"AS": "$38,379,964.65"
},
{
"SG": "$24,134,283.47"
}
]
SQL View structure
My SQL query to return the data
SELECT [CODE],[TOTALVALUE] FROM [dbo].[vw_BuyersByCountryValue]
enter code here
Code for Serializing in ASP.NET
[WebMethod]
public void GetBuyersByCountryValue()
{
using (PMMCEntities ctx = new PMMCEntities())
{
ctx.Configuration.ProxyCreationEnabled = false;
var qry = ctx.vw_BuyersByCountryValue.ToList();
var js = new JavaScriptSerializer();
string strResponse = js.Serialize(qry);
Context.Response.Clear();
Context.Response.ContentType = "application/json";
Context.Response.AddHeader("content-length", strResponse.Length.ToString(CultureInfo.InvariantCulture));
Context.Response.Flush();
Context.Response.Write(strResponse);
HttpContext.Current.ApplicationInstance.CompleteRequest();
}
}

It is very simple
// data from the query
// SELECT CODE, TOTALVALUE FROM vw_BuyersByCountryValue
var sqldata = new []
{
new { Code = "AF", TotalValue = "$23,554,857.27" },
new { Code = "AS", TotalValue = "$38,379,964.65" },
new { Code = "SG", TotalValue = "$24,134,283.47" },
};
var mappeddata = sqldata.Select( r =>
{
var dict = new Dictionary<string,string>();
dict[r.Code] = r.TotalValue;
return dict;
});
var json = JsonConvert.SerializeObject(mappeddata,Formatting.Indented);
content of json
[
{
"AF": "$23,554,857.27"
},
{
"AS": "$38,379,964.65"
},
{
"SG": "$24,134,283.47"
}
]
.net fiddle sample
You can even populate it as
{
"AF": "$23,554,857.27",
"AS": "$38,379,964.65",
"SG": "$24,134,283.47"
}
with
var sqldata = new []
{
new { Code = "AF", TotalValue = "$23,554,857.27" },
new { Code = "AS", TotalValue = "$38,379,964.65" },
new { Code = "SG", TotalValue = "$24,134,283.47" },
};
var mappeddata = sqldata.ToDictionary(r => r.Code, r => r.TotalValue);
var json = JsonConvert.SerializeObject(mappeddata,Formatting.Indented);
.net fiddle sample
Update
[WebMethod]
public void GetBuyersByCountryValue()
{
using (PMMCEntities ctx = new PMMCEntities())
{
ctx.Configuration.ProxyCreationEnabled = false;
var qry = ctx.vw_BuyersByCountryValue.ToList();
var mapped = qry.Select r =>
{
var dict = new Dictionary<string,string>();
dict[r.CODE] = r.TOTALVALUE;
return dict;
});
string strResponse = Newtonsoft.Json.JsonConvert.SerializeObject(mapped);
Context.Response.Clear();
Context.Response.ContentType = "application/json";
Context.Response.AddHeader("content-length", strResponse.Length.ToString(CultureInfo.InvariantCulture));
Context.Response.Flush();
Context.Response.Write(strResponse);
HttpContext.Current.ApplicationInstance.CompleteRequest();
}
}
You need the NuGet package Newtonsoft.Json

how to query data limit in one type using nest elasticsearch

in NEST 2.x, I wrote code to query data like below:
var query = new QueryContainer();
query = query && new TermQuery { Field = "catId", Value = catId };
query = query && new NumericRangeQuery { Field ="price", GreaterThan = 10 };
var request =new SearchRequest<Project>
{
From = 0,
Size = 100,
Query = query,
Sort = new List<ISort>
{
new SortField { Field = "field", Order = SortOrder.Descending },
...
},
Type?? //problem comes here, how to specify type??
}
var response = _client.Search<Project>(request);
There are more than one type in my index, I want to query data in one of type.(just like query one of table data in a database), I hope in the SearchRequest object initializer have a "Type" parameter.

You can specify the indices and types in the constructor for SearchRequest<T>()
var catId = 1;
var query = new QueryContainer(new TermQuery { Field = "catId", Value = catId });
query = query && new NumericRangeQuery { Field = "price", GreaterThan = 10 };
var request = new SearchRequest<Project>("index-name", Types.Type(typeof(Project), typeof(AnotherProject)))
{
From = 0,
Size = 100,
Query = query,
Sort = new List<ISort>
{
new SortField { Field = "field", Order = Nest.SortOrder.Descending },
}
};
var response = client.Search<Project>(request);
would generate the following query
POST http://localhost:9200/index-name/project%2Canotherproject/_search?pretty=true
{
"from": 0,
"size": 100,
"sort": [
{
"field": {
"order": "desc"
}
}
],
"query": {
"bool": {
"must": [
{
"term": {
"catId": {
"value": 1
}
}
},
{
"range": {
"price": {
"gt": 10.0
}
}
}
]
}
}
}

Add new record var LINQ

I am trying to add a new record to the var "issue". I get a list of XXX from a SQL server DB and return in as follows for a jTable grid:
public dynamic XXXList(int CCC)
{
try
{
var issue = db.XXX.ToList().
Select(c => new { DisplayText = c.AAA, Value = c.BBB, c.CCC}).
Where(h => h.HHH == JJJ);
return (new { Result = "OK", Options = issue });
}
catch (Exception ex)
{
return (new { Result = "ERROR", Message = ex.Message });
}
}
The function returns:
{
"$id": "1",
"Result": "OK",
"Options": [
{
"$id": "2",
"DisplayText": "Food and Beverages",
"Value": 4,
"CCC": 4
},
{
"$id": "3",
"DisplayText": "Wrong software versions",
"Value": 5,
"CCC": 4
}
]
}
How can I add another record to the issue var before returning it?
Example:
{
"DisplayText": "new display text",
"Value": 5,
"CCC": 4
}
EDIT:
Here is my function after applying the answers:
public dynamic XXXList(int CCC)
{
try
{
var newRecord = new[] { new { DisplayText = "None", Value = -1, CCC = -1} };
var issue = db.ProjectXXXs.Where(h => h.CCC == JJJ).Select(c => new { DisplayText = c.AAA, Value = c.BBB, c.CCC }).ToList().Concat(newRecord);
return (new { Result = "OK", Options = issue });
}
catch (Exception ex)
{
return (new { Result = "ERROR", Message = ex.Message });
}
}
Thank you very much for all the help.

Concatenate any additional items:
var additional = new[] {new { DisplayText = ..., Value = ..., CCC = ... },
new { DisplayText = ..., Value = ..., CCC = ... }};
var issue = db.XXX.
Where(h => h.HHH == JJJ).
Select(c => new { DisplayText = c.AAA, Value = c.BBB, c.CCC}).
ToList().
Concat(additional);
(EDIT: Credit for the reordering of Where, Select and ToList goes to James Curran.)

Well first thing you want to do is move the ToList() to the end. With the ToList where it was, you'd read every column from every record from the Database, build a list from that, and then search it. With the ToList at the end, you send a query for just those columns of that record to the database, and then build a list from what comes back. You'll also need to move the Where before the Select, so it applies to the XXX records and not the output from the select.
var issue = db.XXX
.Where(h => h.HHH == JJJ)
.Select(c => new { DisplayText = c.AAA, Value = c.BBB, c.CCC})
.ToList();
From here the Add or Concat options suggested by other should work, however, you may have to use a named class instead of an anonomous one.
class OptionValues
{
public string DisplayText {get; set;}
public int Value {get; set;}
public int CCC {get; set;}
}
// :
// :
.Select(c => new OptionValues {DisplayText = c.AAA, Value = c.BBB, CCC= c.CCC})

I'm not sure, but can't you just write a line under var issue:
issue.Add(new { });
with the values that you want of course.