Elasticsearch DeleteByQuery not working, getting 400 bad request - c#

I have the following Nest query to delete all the matching documents, quite straight forward but I am getting 400 bad request on it.
var client = new ElasticClient();
var request = new DeleteByQueryRequest<Type>("my-index")
{
Query = new QueryContainer(
new TermQuery
{
Field = "versionId",
Value = "ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"
}
)
};
var response = client.DeleteByQuery(request);
Assert.IsTrue(response.IsValid);
Thanks for any help.
---------------Update---------------
Request Body
{"query":{"term":{"versionId":{"value":"ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"}}}}
Response Body
{"took":0,"timed_out":false,"_indices":{"_all":{"found":0,"deleted":0,"missing":0,"failed":0}},"failures":[]}
Query in Sense plugin:
GET /my-index/type/_search
{
"query": {
"match": {
"versionId": "ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"
}
}
}
Query Response:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 116,
"max_score": 2.1220484,
"hits": []
...
}}
---------------NEST QUERY--------------
DELETE http://localhost:9200/my-index/component/_query?pretty=true
{
"query": {
"term": {
"versionId": {
"value": "ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"
}
}
}
}
Status: 200
{
"took" : 0,
"timed_out" : false,
"_indices" : {
"_all" : {
"found" : 0,
"deleted" : 0,
"missing" : 0,
"failed" : 0
}
},
"failures" : [ ]
}

It sounds like you may be using Elasticsearch 2.x in conjunction with NEST 2.x. As part of Elasticsearch 2.0, Delete by query was moved out of Elasticsearch core and into a separate plugin that needs to be installed. You can install the plugin using the following command within the Elasticsearch bin directory
bin/plugin install delete-by-query
Starting up the node again, Delete by query should now work as expected.
If you ever need to get more details about why a request has failed, you can inspect the .DebugInformation on the response to get the audit trail for the request.

Related

Actions on Google smart home project can't parse QUERY response

I have the following problem. When I turn on/off my bulb in Google Home app it sends me EXECUTE request and then QUERY request for state check. That behaviour is okay, but the problem is that Actions on Google project can't parse my QUERY response.
For clarification: I don't have problems with C# JSON parsing and any of similar topics. I have problem with that the Actions on Google can't understand my QUERY response. I reponse in correct format according to Google's docs but in the web console I see that error occurs and I don't know what is wrong.
It shows following error in the Google Cloud Console:
{
"insertId": "166g06lg15lgekt",
"jsonPayload": {
"executionLog": {
"executionResults": [
{
"actionResults": [
{
"action": {
"actionType": "STATE_QUERY"
},
"device": {
"deviceType": "OUTLET"
},
"status": {
"externalDebugString": "JSON payload not an object.",
"isSuccess": false,
"statusType": "PARTNER_RESPONSE_INVALID_PAYLOAD"
}
}
],
"executionType": "PARTNER_CLOUD",
"latencyMsec": "191",
"requestId": "4734217757620110233"
}
]
},
"locale": "en-US"
},
"logName": "projects/smartlightproject-f47f4/logs/assistant_smarthome%2Fassistant_smarthome_logs",
"receiveTimestamp": "2022-07-13T13:23:04.088211105Z",
"resource": {
"labels": {
"project_id": "smartlightproject-f47f4"
},
"type": "assistant_action_project"
},
"severity": "ERROR",
"timestamp": "2022-07-13T13:23:04.088211105Z"
}
QUERY request which I receive is
{
"inputs": [
{
"intent": "action.devices.QUERY",
"payload": {
"devices": [
{
"customData": {
"barValue": true,
"bazValue": "foo",
"fooValue": 74
},
"id": "123"
}
]
}
}
],
"requestId": "5460596871498683621"
}
And my response to that QUERY request is
{
"requestId": "5460596871498683621",
"payload": {
"devices": {
"123": {
"on": true,
"online": true
}
}
}
}
I think that everything is as Google wants to in documentation but I can't find the solution nor cause of that issue. I appreciate your help, really.
If it matters I write my local fulfillment in ASP.NET.
That's how I send the QUERY response
var queryObj = JsonConvert.DeserializeObject<dynamic>("{\"requestId\":\"" + requestId + "\",\"payload\":{\"devices\":{\"123\":{\"on\":true,\"online\":true}}}}");
_logger.LogInformation("Odpowiedź na żądanie QUERY." + originalRequest + "\n\n" + JsonConvert.SerializeObject((object)queryObj));
return Ok((object)queryObj);

Just get error messages text from modelstate in web api

I want to return just error messages text in web api in ASP.Net Core 2.2,
I use the following code to return values:
if (!ModelState.IsValid)
{
return BadRequest(ModelState.Values);
}
and i get the following response:
[
{
"childNodes": null,
"children": null,
"key": "user.Username",
"subKey": {
"buffer": "user.Username",
"offset": 5,
"length": 8,
"value": "Username",
"hasValue": true
},
"isContainerNode": false,
"rawValue": null,
"attemptedValue": null,
"errors": [
{
"exception": null,
"errorMessage": "Username is required"
}
],
"validationState": 1
}
]
I need just error messages text for response, like this:
{
'data': [
Username is required,
Password is required,
],
'status': 'error'
}
Well, since you're using ASP.NET Core 2.2, you should really be using the ApiController attribute:
[ApiController]
public class FooApiController : ControllerBase
With that, ASP.NET Core actually takes care of bad requests for you automatically, so you don't even need this check in your action at all.
Otherwise, you should use:
ModelState.ToDictionary(x => x.Key, x => x.Value.Errors);
Or if you just want the errors alone, not keyed to the individual properties:
ModelState.SelectMany(x => x.Value.Errors);

Scrolling and Paging in Elastic/Nest 6+

Intro
I'm upgrading the elastic version to 6.3 (previously we were using 5.4.
Our app is written in C#, thus we use NEST.NET dll to talk with the Elastic server, so we are also updating it to the version 6.0.0.0.
The use case - Before
Until version 5, I was able to execute this query:
jsonStr ="
{
"from": 16224,
"size": 12,
"query": {
"bool": {
"filter": [
{
"bool": {
"must": [
{
"terms": {
"COMPANY": [
"AMP Services Ltd"
]
}
}
]
}
}
]
}
}
}
}"
Using this NEST/C# code:
Func<SearchRequestParameters, SearchRequestParameters> requestParameters = null;
requestParameters = a => a.Scroll(new TimeSpan(0, 1, 0));
response = Connection.Client.GetInstance().LowLevel.Search<dynamic>("myindex", new PostData<dynamic>(jsonStr), requestParameters);
And with that, I was able to fetch the data without problems,
The use case - NOW
Now, with version 6, I'm trying to execute this very same query:
jsonStr ="
{
"from": 16224,
"size": 12,
"query": {
"bool": {
"filter": [
{
"bool": {
"must": [
{
"terms": {
"COMPANY": [
"AMP Services Ltd"
]
}
}
]
}
}
]
}
}
}"
Using this NEST/C# code (as the previus method signatures are no longer available):
SearchRequestParameters searchRequest = new SearchRequestParameters();
searchRequest.Scroll = new TimeSpan(0, 1, 0);
response = Connection.Client.GetInstance().LowLevel.Search<StringResponse>("myindex", PostData.String(jsonStr), searchRequest);
And I'm getting this error: "Validation Failed: 1: using [from] is not allowed in a scroll context;"
Documentation
I could not find anything in here (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html) and here (https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/sliced-scroll-search-usage.html) to help me replace this logic. Nothing in the forums either.
Do you guys have any insights on this?
Thanks
It looks to be related to a validation change in Elasticsearch in 6.0.0; In 5.x, the from parameter was allowed for a scroll request but silently ignored. Now in 6.0.0, Elasticsearch is stricter and validates whether from is present for a scroll request and if it is, returns a bad response with an appropriate validation error.
Since a from parameter doesn't make sense for a scroll request, the solution to this is to do one of these two
Remove the from parameter when using the Scroll API
Continue to use the from parameter but do not use the Scroll API.
As an aside, If you are needing to scroll many documents, you may want to use ScrollAll() observable helper to do so.

C# .net core web api post parameter always null

So this is driving me nuts. I'm doing something very simple sending POST request to my web api. The endpoint is set up as follows:
[HttpPost]
[Route("locations")]
public async Task<IActionResult> PostLocations([FromBody]IEnumerable<Location>locations)
and I'm making the call as follows:
http://localhost:60254/api/Fetch/locations
With the body
{
"Locations": [
{
"LocationId": "111",
"ProductId": 110,
"Sku": "11131-LJK"
}
]
}
And header: content-type: application/json
now again, this is VERY simple something that should work out of the box and this fricking framework change is messing everything up.
Now, if I get the HttpContext and read the body stream directly
using (StreamReader reader = new StreamReader(HttpContext.Request.Body, Encoding.UTF8))
{
string body = reader.ReadToEnd();
}
I can see the body being sent correctly, I have a super well formed json that I can transform into whatever I want. So the question is what am I missing that this endpoint doesn't work?
What configuration the web api project template is not adding out of the box for this to work?
Your payload is not a list of Location but an object with a Locations property that's a list.
Instead of
{
"Locations": [
{
"LocationId": "111",
"ProductId": 110,
"Sku": "11131-LJK"
}
]
}
use
[
{
"LocationId": "111",
"ProductId": 110,
"Sku": "11131-LJK"
}
]
Don't pass a json object, pass a stringified one:
var location = [
{
"LocationId": "111",
"ProductId": 110,
"Sku": "11131-LJK"
}
]
var dataToPost = JSON.stringify(location);
For others...make sure your [FromBody] model and all child classes have parameterless constructors

how to build a good c# data structure

my aim is to develop a custom search based on goeuro.com(overview) in my spare time.
I simplyfied the search parameters to the minimum.
For example(you can try this, as loong as the search_id is valid):
http://www.goeuro.com/GoEuroAPI/rest/api/v5/results?&search_id=428558909
The search_id will be generated when you visit http://www.goeuro.com and enter the first time your search parameters.
This is the simplified data structure I generated from http://json2csharp.com/ using this as my input JSON:
{
"serviceVersion": "v1.0.0",
"sortVariants": null,
"companies": {
"1007": {
"name": "Eurolines Germany",
"logoUrl": "http://cdn-goeuro.com/static_content/web/logos/{size}/eurolines_germany.png",
"id": "1007"
}
},
"outbounds": {
"3624107261930718525-38-flight-1-27": {
"companyId": "38",
"mode": "flight",
"duration": "873",
"outboundId": "3624107261930718525",
"journeyId": "27",
"departureTime": "2017-01-15T19:12:00.000+01:00",
"arrivalTime": "2017-01-16T09:45:00.000+01:00",
"stops": "1",
"price": 16209,
"updatedAt": "1",
"segments": [ 1344486303, 574447503, 689435565, 833161604 ],
"arrivalOvernightOffset": 1,
"overnightOffset": 1
}
},
"query": {
"roundTrip": false,
"airportToAirport": false,
"locationsOutsideEurope": false,
"searchId": "428558909",
"departurePosition": "377001",
"arrivalPosition": "398532",
"departureDate": "2017-01-15T00:00:00.000+01:00",
"passengers": {
"adults": 1,
"children": 0,
"infants": 0
},
"userInfo": {
"userLocale": "en",
"userCurrency": "EUR"
},
"searchModes": {
"bus": {
"status": "done",
"resultsQty": 13,
"filteredResultsQty": 13
},
"flight": {
"status": "done",
"resultsQty": 276,
"filteredResultsQty": 276
},
"train": {
"status": "done",
"resultsQty": 4,
"filteredResultsQty": 4
}
}
},
"itineraries": [
{ "outboundLegId": "3624107261930718525-38-flight-1-27" }
],
"segmentDetails": {
"1002857016": {
"type": "flight",
"departureTime": "2017-01-16T08:35:00.000+01:00",
"arrivalTime": "2017-01-16T12:05:00.000+02:00",
"departurePosition": "313870",
"arrivalPosition": "314920",
"duration": "150",
"company": "190",
"transportId": "ca6199",
"direction": "OUTBOUND",
"overnightOffset": 1,
"departureOvernightOffset": 1,
"arrivalOvernightOffset": 1
}
},
"positions": {
"2091737": {
"positionType": "busstation",
"name": "Warszawa, Warszawa Centralny",
"cityName": "Warsaw",
"latitude": 52.22782,
"longitude": 21.00224
}
},
"currencies": {
"EUR": {
"name": "Euro",
"symbol": "€"
}
}
}
The result from http://json2csharp.com/ is pretty good, but on the other side it generates something like this
public class Outbounds
{
public __invalid_type__362410726193071852538Flight127 __invalid_name__3624107261930718525-38-flight-1-27 { get; set; }
}
I see two problems here:
using __invalid_name__3624107261930718525-38-flight-1-27 is not a valid identifier in c#
and the above mentioned name is some random name, which I can not rely on my data structure.
So, my actually questions are:
How can I handle the request ?
How shall the data structure look like ?
By the way this is the code I am using(plus the generated results from http://json2csharp.com/):
static void Main()
{
var client = new RestClient("https://www.goeuro.com");
var request = new RestRequest("/GoEuroAPI/rest/api/v5/results?&search_id=428558909", Method.GET);
request.RequestFormat = DataFormat.Json;
// contentType: "application/json; charset=utf-8",
var response = client.Execute<Response>(request).Data;
}
Actually I found an existing solution(partly) for my problem:
https://github.com/evgenTraytyak/goeuro-api
, which is actually written in node.js, but I need c#, but by the way this example does not quite work(only if you have an existing search_id) and at the end I want to get the search_id from goeuro, without manually type it to my code.
Maybe the reason that using this is that the JSON-Format changed...

Categories

Resources