I am new to elastic search. I am trying to use the code to search text. getting the Error "{Invalid NEST response built from a unsuccessful () low level call on POST: /foo/_doc}" when trying to run the code. I think I am missing something how to use Nest nuGet any help is welcome
using IdentityServer4.Models;
using Nest;
using System;
using System.Collections.Generic;
using System.Linq;
using static System.Net.Mime.MediaTypeNames;
class Program
{
static void Main(string[] args)
{
var settings = new ConnectionSettings(new Uri("http://127.0.0.1:9200"))
.DefaultIndex("foo");
var client = new ElasticClient(settings);
var indexName = "indexName";
if (client.IndexExists(indexName).Exists)
{
client.DeleteIndex(indexName);
}
var createIndexResponse = client.CreateIndex(indexName, c => c
.Settings(s => s
.Analysis(a => a
.Analyzers(an => an
.Custom("my_analyzer", ca => ca
.Tokenizer("standard")
.CharFilters("html_strip")
.Filters("lowercase")
)
)
)
)
.Mappings(m => m
.Map<Document>(mp => mp
.AutoMap()
.Properties(p => p
.Text(t => t
.Name(n => n.Title)
.Analyzer("my_analyzer")
)
.Text(t => t
.Name(n => n.Content)
.Analyzer("my_analyzer")
)
)
)
)
);
var doc = new Document
{
Title = "Friends are friendlier friendlies that are friendly and classify the friendly classification class.",
// Remove stop word like are, and, that,the
Content = "Flowery flowers flow through following the flower flows."
};
var indexResponse = client.IndexDocument(doc);
// Error occurred here
Console.WriteLine(indexResponse.Result);
Console.ReadKey();
}
}
Document class
class Document
{
public string Title { get; set; }
public string Content { get; set; }
}
Debugging Information
"Invalid NEST response built from a unsuccessful () low level call on PUT: /indexName\r\n# Audit trail of this API call:\r\n - [1] ProductCheckOnStartup: Took: 00:00:02.0264861\r\n - [2] ProductCheckFailure: Node: http://127.0.0.1:9200/ Took: 00:00:02.0261849\r\n# OriginalException: Elasticsearch.Net.ElasticsearchClientException: The client is unable to verify that the server is Elasticsearch due to an unsuccessful product check call. Some functionality may not be compatible if the server is running an unsupported product. Call: Status code unknown from: GET /\r\n ---> Elasticsearch.Net.PipelineException: The client is unable to verify that the server is Elasticsearch due to an unsuccessful product check call. Some functionality may not be compatible if the server is running an unsupported product.\r\n ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it.\r\n ---> System.Net.Sockets.SocketException (10061): No connection could be made because the target machine actively refused it.\r\n at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)\r\n --- End of inner exception stack trace ---\r\n at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)\r\n at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)\r\n at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)\r\n at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)\r\n at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)\r\n at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)\r\n at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)\r\n at Elasticsearch.Net.HttpConnection.Request[TResponse](RequestData requestData)\r\n --- End of inner exception stack trace ---\r\n at Elasticsearch.Net.RequestPipeline.ThrowIfTransientProductCheckFailure()\r\n at Elasticsearch.Net.RequestPipeline.Ping(Node node)\r\n at Elasticsearch.Net.Transport1.Ping(IRequestPipeline pipeline, Node node)\r\n at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)\r\n --- End of inner exception stack trace ---\r\n# Request:\r\n<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>\r\n# Response:\r\n<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>\r\n"
Related
What I'm trying to do is to create only one saga instance, not updated by subsequent events. So for example if I send EventA at 09:00, with CorrelationId equal to 979158a2-dfa0-45f0-9107-8b8b8028bb9f and then the saga for that event is still alive at 10:00, then if at 10:00 I send another instance of EventA with the same CorrelationId that was sent previously, I don't want it to overwrite the previous saga state but instead I would like this event to be ignored by MT. I kind of achieved that, because my saga state is not overriden (only its version is incremented), but the problem is that MT throws an error when the new event arrives. The exception is (+one log entry above it):
[18:41:12 DBG] SAGA:App.Services.Communicators.Amazon.Dispatcher.Api.Sagas.GetOrdersState:0368a723-a819-450f-b6e5-9211d1d6a3a9 Dupe App.Services.Financial.Contracts.GetOrdersCommand
MongoDB.Driver.MongoWriteException: A write operation resulted in an error.
E11000 duplicate key error collection: Dispatcher.get.orders.states index: _id_ dup key: { _id: BinData(3, 23A7680319A80F45B6E59211D1D6A3A9) }
---> MongoDB.Driver.MongoBulkWriteException`1[App.Services.Communicators.Amazon.Dispatcher.Api.Sagas.GetOrdersState]: A bulk write operation resulted in one or more errors.
E11000 duplicate key error collection: Dispatcher.get.orders.states index: _id_ dup key: { _id: BinData(3, 23A7680319A80F45B6E59211D1D6A3A9) }
at MongoDB.Driver.MongoCollectionImpl`1.BulkWriteAsync(IClientSessionHandle session, IEnumerable`1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionBase`1.InsertOneAsync(TDocument document, InsertOneOptions options, Func`3 bulkWriteAsync)
--- End of inner exception stack trace ---
at MongoDB.Driver.MongoCollectionBase`1.InsertOneAsync(TDocument document, InsertOneOptions options, Func`3 bulkWriteAsync)
at MassTransit.MongoDbIntegration.Saga.Context.MongoDbSagaRepositoryContext`2.Insert(TSaga instance)
And the important part of the saga config:
Event(() => GetOrdersIntegrationEventReceived, e =>
e
.CorrelateById(x => x.Message.CorrelationId)
.SelectId(x => x.Message.CorrelationId)
.SetSagaFactory(ctx => new GetOrdersState
{
CorrelationId = ctx.Message.CorrelationId,
PartnerId = ctx.Message.PartnerId,
LastUpdatedBeforeFromRequest = ctx.Message.LastUpdatedBefore
})
.InsertOnInitial = true);
Is what I'm trying to do even possible?
MassTransit will only create one instance of the saga for each CorrelationId, so your reasoning is correct. However, you're approach is a little off and could use some tweaking.
For instance, your event configuration:
Event(() => GetOrdersIntegrationEventReceived, e => e.CorrelateById(x => x.Message.CorrelationId));
That's it, all the extra stuff is why you're seeing messages in the debug log.
Then, after the event is declared:
During(Initial, Created,
When(GetOrdersIntegrationEventReceived)
.Then(context =>
{
context.Instance.PartnerId = context.Message.PartnerId,
context.Instance.LastUpdatedBeforeFromRequest = context.Message.LastUpdatedBefore
})
.TransitionTo(Created)
);
That's it.
I am using Visual Studio for Mac for a .NET Console C# project using Google Speech API.
I am getting an error that says:
Grpc.Core.RpcException: 'Status(StatusCode=Unavailable, Detail="Getting metadata from plugin failed with error: Exception occurred in metadata credentials plugin. Google.Apis.Auth.OAuth2.Responses.TokenResponseException: Error:"invalid_grant", Description:"Robot is disabled.", Uri:""
at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions.ExecuteAsync(TokenRequest request, HttpClient httpClient, String tokenServerUrl, CancellationToken taskCancellationToken, IClock clock) in C:\Apiary\support1351\Src\Support\Google.Apis.Auth\OAuth2\Requests\TokenRequestExtenstions.cs:line 52
at Google.Apis.Auth.OAuth2.ServiceAccountCredential.RequestAccessTokenAsync(CancellationToken taskCancellationToken) in C:\Apiary\support1351\Src\Support\Google.Apis.Auth\OAuth2\ServiceAccountCredential.cs:line 212
at Google.Apis.Auth.OAuth2.TokenRefreshManager.RefreshTokenAsync() in C:\Apiary\support1351\Src\Support\Google.Apis.Auth\OAuth2\TokenRefreshManager.cs:line 129
at Google.Apis.Auth.OAuth2.TokenRefreshManager.GetAccessTokenForRequestAsync(CancellationToken cancellationToken) in C:\Apiary\support1351\Src\Support\Google.Apis.Auth\OAuth2\TokenRefreshManager.cs:line 114
at Google.Apis.Auth.OAuth2.ServiceAccountCredential.GetAccessTokenForRequestAsync(String authUri, CancellationToken cancellationToken) in C:\Apiary\support1351\Src\Support\Google.Apis.Auth\OAuth2\ServiceAccountCredential.cs:line 235
at Grpc.Auth.GoogleAuthInterceptors.<>c__DisplayClass2_0.<<FromCredential>b__0>d.MoveNext() in T:\src\github\grpc\src\csharp\Grpc.Auth\GoogleAuthInterceptors.cs:line 48
--- End of stack trace from previous location where exception was thrown ---
at Grpc.Core.Internal.NativeMetadataCredentialsPlugin.GetMetadataAsync(AuthInterceptorContext context, IntPtr callbackPtr, IntPtr userDataPtr) in T:\src\github\grpc\src\csharp\Grpc.Core\Internal\NativeMetadataCredentialsPlugin.cs:line 83")'
at the line I mark with the comment "// error here" in the code:
class Program
{
static void Main(string[] args)
{
var URI = "https://speech.googleapis.com/v1/speech:recognize?key=[API_KEY]";
Console.WriteLine("Start!");
AsyncRecognizeGcs(URI);
Console.WriteLine("End.");
}
static object AsyncRecognizeGcs(string storageUri)
{
var speech = SpeechClient.Create();
var longOperation = speech.LongRunningRecognize(new RecognitionConfig()
{
Encoding = RecognitionConfig.Types.AudioEncoding.Flac,
SampleRateHertz = 44100,
AudioChannelCount = 2,
LanguageCode = "en",
}, RecognitionAudio.FromStorageUri(storageUri)); // error here
longOperation = longOperation.PollUntilCompleted();
var response = longOperation.Result;
foreach (var result in response.Results)
{
foreach (var alternative in result.Alternatives)
{
Console.WriteLine($"Transcript: { alternative.Transcript}");
}
}
return 0;
}
}
How do I fix this? Is there something else I need to set up with Google?
Recommend updating to a new version of Google.Cloud.PubSub.V1, which will pull in a newer version of Grpc.Core.
Use the last update of Google.Apis.Auth.
It turns out that I had the wrong credentials json file. Stupid mistake.
I have a continuous WebJob which executes a function which is queue triggered and it occasionally throws System.Threading.Tasks.TaskCanceledException.
WebJobs SDK version : 2.1.0
StackTrace:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: Functions.ProcessCreationQueueMessage --->
System.AggregateException : One or more errors occurred. ---> A task was canceled. at
System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at
System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task`1.get_Result() at
Normandy.Portal.Reporting.Creation.WebJobs.Functions.ProcessCreationQueueMessage(String message,String& outputQueueMessage,TextWriter log) at
D:\a\1\s\Portal\src\Normandy.Portal.Reporting.Creation.WebJobs\Functions.cs : 105 at lambda_method(Closure ,Functions ,Object[] ) at Microsoft.Azure.WebJobs.Host.Executors.VoidMethodInvoker`2.InvokeAsync(TReflected instance,Object[] arguments) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeAsync(IFunctionInvoker invoker,ParameterHelper parameterHelper,CancellationTokenSource timeoutTokenSource,CancellationTokenSource functionCancellationTokenSource,Boolean throwOnTimeout,TimeSpan timerInterval,IFunctionInstance instance) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstance instance,ParameterHelper parameterHelper,TraceWriter traceWriter,CancellationTokenSource functionCancellationTokenSource)
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??)
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) ---> (Inner Exception #0) System.Threading.Tasks.TaskCanceledException : A task was canceled.<--- End of inner exception
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance,CancellationToken cancellationToken)
WebJob:
public void ProcessCreationQueueMessage([QueueTrigger("reportcreation")] string message, [Queue("reportretrieval")] out string outputQueueMessage, TextWriter log)
{
JsonSerializerSettings settings = new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All };
log.WriteLine(message);
...
try
{
var jobId = _reportingService.RequestReport(req.ReportName, searchString).Result;
var output = new RetrieveReportDataRequest()
{
CreationRequest = new ReportCreationRequest()
{
UserId = req.UserId,
UID = req.UID,
CreateDate = req.CreateDate,
IsMultiSchoolUser = req.IsMultiSchoolUser,
CentreCode = req.CentreCode,
ReportName = req.ReportName,
InputSearchParameters = req.InputSearchParameters,
},
SplunkSearchParameters = searchString,
SplunkJobId = jobId,
Created = DateTime.UtcNow,
};
outputQueueMessage = JsonConvert.SerializeObject(output, settings);
}
catch (Exception ex)
{
log.WriteLine(ex.Message);
throw;
}
}
There is no any time taking tasks apparently. This is failing once for certain messages. What am I missing?
It can be related to host-settings - look at Azure Portal -> AppService configuration, make sure that Always On is set to On:
I'm using Azure Service Bus Topics with the AMQP protocol in the West Europe datacenter.
This is a schematic way of how the solution implented works:
private Microsoft.Azure.ServiceBus.SubscriptionClient CreateClient() {
string serviceBusConnectionString;
strin serviceBusTopicName;
string subscriptionName;
var subscriptionClient = new Microsoft.Azure.ServiceBus.SubscriptionClient(serviceBusConnectionString, serviceBusTopicName, subscriptionName) {
PrefetchCount = 0
};
return subscriptionClient;
}
public async Task<ISubscriptionClient> CreateSubscriptionClientAsync() {
//Some logic on the subscriptionClient, caching, creating a new one if it doesn't exists, etc.
return CreateClient()
}
private async Task CallbackAsync(Message msg, CancellationToken cancellationToken) {
//Do stuff with the Message
//when you're done Completethe message
}
public async Task<string> OpenAsync(CancellationToken cancellationToken) {
subscriptionClient = await CreateSubscriptionClientAsync().ConfigureAwait(false);
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandlerAsync) {
AutoComplete = false,
MaxAutoRenewDuration = TimeSpan.FromHours(8)
};
subscriptionClient.RegisterMessageHandler(CallbackAsync, messageHandlerOptions);
return string.Empty;
}
But last night I had couple of thousand exception like this one:
Exception: "System.InvalidOperationException: The AMQP object g2b-sessionXXXXXXXX is closing. Operation 'attach' cannot be performed.
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<OnReceiveAsync>d__86.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<>c__DisplayClass64_0.<<ReceiveAsync>b__0>d.MoveNext()
I'm using the Microsoft.Azure.ServiceBus v3.1.0.0,
On line I didn't find anything useful about this "Operation 'attach' cannot be performed".
On the message bus exception list page I didn't find any information about this specific problem. And in the status history site there is no reference about any outage involving Service Bus.
Did anyone experience this problem before?
What is causing the exception?
Do I need to implement any retry logic? How?
Any help is apreciated, thanks.
I have windows service that is basically reading messages from customer and we do process and sends them over HTTP/TCP/File etc. Service is written in C#, for database interaction I use NHibernate and TPL task.
For every batch of message service reads and inserts into SQL server database in separate TPL Task and same messages is then fetched from database in another TPL task we send them over HTTP/TCP/File after processing, and we also save this records into database using NHibernate ISession.
piece of code is below
public Func<ISession> GetSession { get; set; }
[Transaction]
public string SaveInMessage(ISession session, string message)
{
try
{
using (var session = GetSession()){
session.Transaction.Begin();
var inMessage = new InMessage();
var task = new Task(()=> InsertToDatabase(session, inMessage));
session.Transaction.Commit();
}
}
catch(Exception ex)
{
session.Transaction.Rollback();
}
}
public void InsertToDatabase(ISession session, InMessage inMessage){
session.SaveOrUpdate(inMessage);
}
[Transaction]
public bool SaveOutMessage()
{
try
{
using (var session = GetSession()){
session.Transaction.Begin();
var inMessage = session.Load<InMessage>();
var outMessage = new OutMessage();
var task = new Task(()=> ConvertMessage(outMessage, inMessage, session));
var task = new Task(()=> SendMessage(outMessage, session, outProtocol));
session.Transaction.Commit();
}
}
catch(Exception ex)
{
session.Transaction.Rollback();
}
}
public void ConvertMessage(OutMessage outMessage, ISession session, Http url)
{
conversion logic goes here;
}
public void SendMessage(OutMessage outMessage,ISession session,Protocol outProtocol)
{
Sending message logic goes here;
session.SaveOrUpdate(inMessage);
}
So, in above I have used Castle.Windsor for IoC and Transaction attribute used in these two methods are from same.
I am keep getting below AggregateException along with OutOfMemoryException within NHibernate and TPL.
System.AggregateException: One or more errors occurred. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.String.ConcatArray(String[] values, Int32 totalLength)
at System.String.Concat(Object[] args)
at NHibernate.Engine.Cascade.CascadeOn(IEntityPersister persister, Object parent, Object anything)
at NHibernate.Event.Default.AbstractFlushingEventListener.CascadeOnFlush(IEventSource session, IEntityPersister persister, Object key, Object anything)
at NHibernate.Event.Default.AbstractFlushingEventListener.PrepareEntityFlushes(IEventSource session)
at NHibernate.Event.Default.AbstractFlushingEventListener.FlushEverythingToExecutions(FlushEvent event)
at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event)
at NHibernate.Impl.SessionImpl.Flush()
at NHibernate.Transaction.AdoTransaction.Commit()
at Service.ProcessMessage(Message message, ISession session) in C:\Project\Service\ProcessMessage.cs:line 247
at Service.ProcessMessage.<>c__DisplayClass22_0.<SendMessages>b__0(Task c) in C:\Project\Service\ProcessMessage.cs:line 74
at System.Threading.Tasks.ContinuationTaskFromTask.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait(CancellationToken cancellationToken)
at Service.ProcessMessage.SendMessages(CancellationToken cancelToken) in C:\Project\Service\ProcessMessage.cs:line 73
---> (Inner Exception #0) System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.String.ConcatArray(String[] values, Int32 totalLength)
at System.String.Concat(Object[] args)
at NHibernate.Engine.Cascade.CascadeOn(IEntityPersister persister, Object parent, Object anything)
at NHibernate.Event.Default.AbstractFlushingEventListener.CascadeOnFlush(IEventSource session, IEntityPersister persister, Object key, Object anything)
at NHibernate.Event.Default.AbstractFlushingEventListener.PrepareEntityFlushes(IEventSource session)
at NHibernate.Event.Default.AbstractFlushingEventListener.FlushEverythingToExecutions(FlushEvent event)
at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event)
at NHibernate.Impl.SessionImpl.Flush()
at NHibernate.Transaction.AdoTransaction.Commit()
at Service.ProcessMessage.Process(Message message, ISession session) in C:\Project\Service\ProcessMessage.cs:line 247
at Service.ProcessMessage.<>c__DisplayClass22_0.<SendMessages>b__0(Task c) in C:\Project\Service\ProcessMessage.cs:line 74
at System.Threading.Tasks.ContinuationTaskFromTask.InnerInvoke()
at System.Threading.Tasks.Task.Execute()<---
LogExtension.Event => LogExtension.Event => LogExtension.LogEvent
Above error is coming after 4-5 hours after windows service is started.
Any hint or help is appreciated.
.Net framework version I use is 4.5
NHibernate version is 4.0.4.4000
Castle.Windsor version is 3.3.0
FluentNhibernate version is 1.3.0.733
Castle.Facilities.NHibernate version is 0.7.1.23602
Castle.Facilities.AutoTx version is 3.2.207.2207
Castle.Core version is 3.3.3
You haven't posted your logic for GetSession(), but I get the impression you are re-using the same session for all of the messages. Sessions are supposed to be short lived. Keeping a single session open will result in the first level cache growing to the point where you get performance degradation and eventually run out of memory.