Elasticsearch NEST update by script - c#

I'm trying to update some fields by script. In Postman I'm sending the following request:
https://search.test.com/items_example/item/en_01_2/_update
the request's body:
{
"script": {
"inline": "ctx._source.Title = params.Title; ctx._source.Desc = params.Desc",
"params": {
"Title": "New Title",
"Desc": "New Desc"
}
}
}
But I have no idea how I can send this request using NEST. Can someone please help me?
Elasticsearch 5.4.1 version, NEST 5.6.1

Update it with your index settings & query
var elasticClient = new ElasticClient(settings);
var scriptParams = new Dictionary<string, object>
{
{"Title", "New Title"},
{"Desc", "New Desc"}
};
var response = elasticClient
.UpdateByQuery<dynamic>(q => q.Query(rq => rq.Term(....))
.Script(script =>
script.Inline(
$"ctx._source.Title = params.Title;" +
$"ctx._source.Desc = params.Desc ;"
)
.Params(scriptParams));
Edit: If you're looking for just Update, just change the syntax to
var response = elasticClient.Update<dynamic>(
"items_example/item/en_01_2" //This is your document path
, request => request.Script(
script =>
script.Inline(
$"ctx._source.Title = params.Title;" +
$"ctx._source.Desc = params.Desc ;"
)
.Params(scriptParams)));

If someone is here looking for the solution with NEST 6x version, please see below
public async Task UpdateId(MetaData request)
{
try
{
var scriptParams = new Dictionary<string, object>
{
{ "Id",request.Id}
};
var script = $"ctx._source.Id= params.Id;";
var indexResponse =
await EsClient.UpdateByQueryAsync<Doc>(
qd => qd.Index(request.IndexName)
.Conflicts(Conflicts.Proceed)
.Query(
rq => rq.Term("_id", request._id))
.Script(
s => s.Source(script)
.Params(scriptParams)
)
)
);
if (!indexResponse.IsValid)
{
}
}
catch (Exception ex)
{
}
}
[ElasticsearchType(Name = "_doc")]
public class Doc
{
}

Related

SwaggerRequestExample not showing on Swagger UI

I've been trying to document this API with Swashbuckle.Example. I have followed this tutorial. The SwaggerResponseExample attribute works but the SwaggerRequestExample is not rendered. I have also checked the json in /docs/v1 and again the Response Example was present but not the Request Example.
What's going on?
How do I debug this?
Controller.cs
[System.Web.Http.HttpPost]
[SwaggerOperation(OperationId = "SAAT_ciclotarifario_GetById")]
[SwaggerResponseExample(HttpStatusCode.OK, typeof(GrupoFtFronteiraPostResponseExamples))]
[SwaggerRequestExample(typeof(GrupoFtFronteira), typeof(GrupoFtFronteiraPostRequestExamples))]
[ActionName("GrupoFtFronteira")]
public HttpResponseMessage PostGrupoFtFronteira([FromBody] GrupoFtFronteira requestBody)
{
try
{
var novoGrupoFtFronteira = _grupofronteiraNegocio.PostGrupoFtFronteira(requestBody);
return new HttpResponseMessage(HttpStatusCode.OK)
{
ReasonPhrase = "Success",
Content = new StringContent("idGrupoFtFronteira: " + novoGrupoFtFronteira.IdGrupoftfronteira, System.Text.Encoding.UTF8, "application/json")
};
}
catch (Exception e)
{
var msg = new HttpResponseMessage(HttpStatusCode.InternalServerError)
{
ReasonPhrase = "Internal Server Error",
Content = new StringContent("Erro ao tentar atualizar dados. " + e.Message, System.Text.Encoding.UTF8, "application/json")
};
throw new HttpResponseException(msg);
}
}
SwaggerConfig.cs
public class SwaggerConfig
{
public static void Register()
{
var thisAssembly = typeof(SwaggerConfig).Assembly;
GlobalConfiguration.Configuration
.EnableSwagger(c =>
{
c.ResolveConflictingActions(apiDescriptions => apiDescriptions.First());
c.DocumentFilter<SwaggerFilterOutControllers>();
c.SingleApiVersion("v1", "ons.sact.servico.controller");
c.ApiKey("Authorization")
.Description("JWT token")
.Name("Bearer")
.In("header");
c.OperationFilter<ExamplesOperationFilter>();
})
.EnableSwaggerUi(c =>
{
c.EnableApiKeySupport("apiKey", "header");
});
}
}
I found it!
It seems that I was supposed to fill the Request Type inside this foreach ( KeyValuePair<string, Swashbuckle.Swagger.Schema> definition in schemaRegistry.Definitions)
There's probably a better way around this, since this is a hardcoded solution...
Anyway, thanks a lot for those who helped me out!
SwaggerFilterOutControllers.cs
void IDocumentFilter.Apply(SwaggerDocument swaggerDoc, SchemaRegistry schemaRegistry, IApiExplorer apiExplorer)
{
var models = new List<string>();
foreach (ApiDescription apiDescription in apiExplorer.ApiDescriptions)
{
Console.WriteLine(apiDescription.Route.RouteTemplate);
if (apiDescription.RelativePathSansQueryString().StartsWith("api/v1/")
&& !(apiDescription.RelativePathSansQueryString().StartsWith("api/v1/SAAT/ciclotarifario")
|| apiDescription.RelativePathSansQueryString().StartsWith("api/v1/SAAT/GrupoFtFronteira")))
{
string path = "/" + apiDescription.RelativePathSansQueryString();
swaggerDoc.paths.Remove(path);
}
}
foreach ( KeyValuePair<string, Swashbuckle.Swagger.Schema> definition in schemaRegistry.Definitions)
{
Console.WriteLine(definition.Key);
if (!(definition.Key.Equals("CiclotarifarioPostRequest") ||
definition.Key.Equals("GrupoFtFronteira")
))
{
models.Add(definition.Key.ToString());
}
}
foreach (string modelo in models)
{
swaggerDoc.definitions.Remove(modelo);
}
CreateHeadersDict();
var allOtherPaths = swaggerDoc.paths
.Select(entry => entry.Value)
.ToList();
foreach (var path in allOtherPaths)
{
AddHeadersToPath(path, HeaderType.TokenAuth);
}
}

Tag not set with generate_presigned_post

TL;DR Tags are not being set on the uploaded S3 object while using a generate_presigned_post
Hello Folks,
I'm creating presigned upload urls using Python and Boto3 with the following script:
fields = {
...
"x-amz-meta-u1": "value1",
"x-amz-tagging": "foo=bar"
}
conditions = [
{"acl": "bucket-owner-full-control"},
{"x-amz-meta-u1": "value1"},
{"x-amz-tagging": "foo=bar"},
["content-length-range", 1, 104857600 ]
]
url = s3_client.generate_presigned_post(etl_bucket_name, s3_key, fields, conditions, 3600)
On the client side, I have this code:
var xAmzTaggingPair = fields.FirstOrDefault(pair => pair.Key == "x-amz-tagging");
if (xAmzTaggingPair.Key != null)
{
httpClient.DefaultRequestHeaders.Add(xAmzTaggingPair.Key, xAmzTaggingPair.Value);
}
httpClient.BaseAddress = new Uri(signedUrl);
var formContent = new MultipartFormDataContent();
fields.ToList().ForEach(pair => formContent.Add(new StringContent(pair.Value), pair.Key));
formContent.Add(new StreamContent(fs), "file", Path.GetFileName(filePath));
var response = Task.Run(async () => await httpClient.PostAsync(signedUrl, formContent).ConfigureAwait(false));
var responseResult = response.Result;
S3 doesn't throw an error on the upload, but when I view the object in the console, I don't see the tags being set. What am I missing? The x-amz-meta-u1 metadata IS being set when looking at the console.
Any help is greatly appreciated.

Flurl PostUrlEncoded does GET instead of POST

I must be missing something very obvious, but I can't tell what. I have a DoLoginAsync like so:
private async Task DoLoginAsync(bool force = false)
{
try
{
if (client.Cookies.ContainsKey("user_credentials") && !force)
{
return;
}
var html = client.Request("login").GetStringAsync().Result;
var doc = new HtmlDocument();
doc.LoadHtml(html);
var csrf_token = doc.DocumentNode.SelectNodes("//meta[#name='csrf-token']").First().GetAttributeValue("content", string.Empty);
var values = new Dictionary<string, string>
{
{ "user_session[email]", user },
{ "user_session[password]", password },
{ "authenticity_token", csrf_token }
};
var result = await client.Request("user_session").PostUrlEncodedAsync(values);
}
catch (Exception e)
{
}
When I run this code in a test with a breakpoint in the catch clause I get an exception
Call failed with status code 404 (Not Found): GET http://www.whatever.com/user_session
WTF? I'm expecting PostUrlEncodedAsync to do a POST, not a GET. Anybody have an idea why this can happen?
The Flurl client is instantiated as client = new FlurlClient(BASE_URL).EnableCookies();
UPDATE
Tried the following test which fails with the same exception
[TestMethod]
public async Task TheTest()
{
var message = "";
try
{
var client = new FlurlClient("http://www.slimmemeterportal.nl/").EnableCookies();
var html = await client.Request("login").GetStringAsync();
var doc = new HtmlDocument();
doc.LoadHtml(html);
var csrf_token = doc.DocumentNode.SelectNodes("//meta[#name='csrf-token']").First().GetAttributeValue("content", string.Empty);
var values = new Dictionary<string, string>
{
{ "user_session[email]", "******" },
{ "user_session[password]", "******" },
{ "commit", "inloggen" }, // Not sure if this is actually needed, but it is in the website's request parameters.
{ "authenticity_token", csrf_token }
};
var result = await client.Request("user_session").PostUrlEncodedAsync(values);
}
catch (FlurlHttpException ex)
{
message = ex.Message;
}
Assert.AreEqual("Call failed with status code 404 (Not Found): POST http://www.slimmemeterportal.nl/user_session", message);
}
Mystery solved: As it turns out after some debugging with Wireshark, the website was returning HTTP status code 301. As explained here the default action is to follow the URI in the response's location header using a GET even if the original request was a POST.

Refit / HttpClient / ModernHttpClient - 400 Bad Request

Recently started with ReactiveUI and so far so good. While trying to figure out stuff I came across a HttpClient / Refit problem. I have a long list and every item in that list have to make a request to a API as follows.
This is a ReactiveObject
public SearchResult()
{
LoadStats = ReactiveCommand.CreateAsyncTask(_ => DownloadMultipleCalls(this.IdText, this.DisplayText));
LoadStats
.ThrownExceptions
.SubscribeOn(RxApp.MainThreadScheduler)
.Subscribe(ex => UserError.Throw("Couldn't load stats", ex));
_avaText = LoadStats.Select(x => x.Ava).ToProperty(this, x => x.AvaText, string.Empty);
_perText = LoadStats.Select(x => x.Per).ToProperty(this, x => x.PerText, string.Empty);
this.WhenAnyValue(x => x.IdText)
.Where(x => !string.IsNullOrWhiteSpace(x))
//.Select(_ => Observable.Timer(DateTimeOffset.MinValue, TimeSpan.FromSeconds(1), RxApp.MainThreadScheduler))
.Retry(5)
.InvokeCommand(LoadStats);
}
In DownloadMultipleCalls the following is happening.
async static Task<AvaPer> DownloadMultipleCalls(string id, string displaytext)
{
AvaPer response = new AvaPer();
var block = new ActionBlock<string>(async service =>
{
response = await DownloadCall(service, displaytext);
},
new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 4 });
block.Post(id);
block.Complete();
await block.Completion;
return response;
}
async static Task<AvaPer> DownloadCall(string id, string displaytext)
{
System.Diagnostics.Debug.WriteLine($"Starting download at {DateTimeOffset.Now}");
var client = new HttpClient(NetCache.UserInitiated)
{
BaseAddress = new Uri("https://api.adres.com"),
};
//client.Timeout = TimeSpan.FromMilliseconds(10);
var token = GetToken;
client.DefaultRequestHeaders.Add("authToken", token);
var api = RestService.For<IMyApi>(client);
AvaPer idEntries = new AvaPer();
try
{
var searchResult = await api.GetAvaPerStatusRaw(id); // <== Here is the error (400 - Bad Request)
//var parsedEntries = Task.Run(() => {
idEntries = new AvaPer
{
Id = searchResult.Id.ToString() ?? string.Empty,
display = displaytext ?? string.Empty,
Ava = searchResult.AvailabilityStatus ?? string.Empty,
Per = searchResult.PerformanceStatus ?? string.Empty,
Updated = DateTimeOffset.Now.ToLocalTime().ToString() ?? string.Empty
};
//}).ConfigureAwait(false);
//return idEntries;
}
catch (ApiException ex)
{
var statusCode = ex.StatusCode;
var error = ex.GetContentAs<Models.ServiceModel.ErrorResponse>();
}
return idEntries;
}
Finally it goes all wrong with this line of code var searchResult = await api.GetAvaPerStatusRaw(id); The app crashes with a response "400 Bad Request" and if I look into the error it sais that the there is no id value inplace of the GET request. But when I check the initial value of id there is a value so it should not respond with that. Can anyone help me with this? If you have any question, please do not hesitate to ask. Thank you very much.
Kind regards,
Fernando

How to create async http requests in a loop?

Yesterday I've found out how to create several async http requests without async/await. But today I need to do it in a loop: if some of responses don't satisfy some condition - I need to change a request for them and send these requests again. It may be repeated several times.
I've tried this code:
do
{
var loadingCoordinatesTasks = new List<Task<Terminal>>();
var totalCountOfTerminals = terminalPresetNode.ChildNodes.Count;
var uiTaskScheduler = TaskScheduler.FromCurrentSynchronizationContext();
foreach (var terminal in terminals.Except(_terminalsWithCoordinates))
{
var address = terminal.GetNextAddress();
var webRequest = (HttpWebRequest)WebRequest.Create(GeoCoder.GeoCodeUrl + address);
var webRequestTask = Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse,
webRequest.EndGetResponse,
terminal);
var parsingTask = webRequestTask.ContinueWith(antecedent =>
{
// Parse the response
});
loadingCoordinatesTasks.Add(parsingTask);
}
Task.Factory.ContinueWhenAll(loadingCoordinatesTasks.ToArray(), antecedents =>
{
foreach (var antecedent in antecedents)
{
var terminalWithCoordinates = antecedent.Result;
if (antecedent.Status == TaskStatus.RanToCompletion &&
!terminalWithCoordinates.Coordinates.AreUnknown)
{
_terminalsWithCoordinates.Add(terminalWithCoordinates);
_countOfProcessedTerminals++;
}
}
});
} while (_countOfProcessedTerminals < totalCountOfTerminals);
but is it possible to check the condition in while just after every single set of requests executed?
You can perform the check after increasing the count:
_countOfProcessedTerminals++;
if (_countOfProcessedTerminals >= totalCountOfTerminals)
{
break;
}
Is _countOfProcessedTerminals thread-safe though?
I manage to do it using recursion:
public void RunAgainFailedTasks(IEnumerable<Task<Terminal>> tasks)
{
Task.Factory.ContinueWhenAll(tasks.ToArray(), antecedents =>
{
var failedTasks = new List<Task<Terminal>>();
foreach (var antecedent in antecedents)
{
var terminal = antecedent.Result;
// Previous request was failed
if (terminal.Coordinates.AreUnknown)
{
string address;
try
{
address = terminal.GetNextAddress();
}
catch (FormatException) // No versions more
{
continue;
}
var getCoordinatesTask = CreateGetCoordinatesTask(terminal, address);
failedTasks.Add(getCoordinatesTask);
}
else
{
_terminalsWithCoordinates.Add(terminal);
}
}
if (failedTasks.Any())
{
RunAgainFailedTasks(failedTasks);
}
else
{
// Display a map
}
}, CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
private Task<Terminal> CreateGetCoordinatesTask(Terminal terminal, string address)
{
var webRequest = (HttpWebRequest)WebRequest.Create(GeoCoder.GeoCodeUrl + address);
webRequest.KeepAlive = false;
webRequest.ProtocolVersion = HttpVersion.Version10;
var webRequestTask = Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse,
webRequest.EndGetResponse,
terminal);
var parsingTask = webRequestTask.ContinueWith(webReqTask =>
{
// Parse the response
});
return parsingTask;
}

Categories

Resources