I have several .Net 5.0 microservices with RabbitMQ as message broker. Right now I am switching to AWS SQS. Few services are listening the same message (this is done with Exchange in RabbitMQ). In AWS this could be implemented by subscribing SQS Queue to SNS topic. I created SNS fifo topic and SQS fifo queues, subscribed those queues to topic. When I publish message directly to queue everything works immediately, but when I publish message to SNS topic it takes more than 7 minutes to get message from queue subscribed to topic. Does anybody noticed such huge delay ? Maybe latency?
Btw, all the service are running on aws the same region.
Any ideas ? Will appreciate any advice!
thank you all for your comments and help! In my case lack of experience with sqs was the main problem. I found parameter in queue settings "Delivery delay" and set smaller value and now I receive messages immediately.
Related
Using MassTransit with AWS SQS (simple queue service), we would like to implement a heartbeat, as advised in the AWS docs:
If you don't know how long it takes to process a message, create a heartbeat for your consumer process.
This requires changing the visibility of the message from code. AWS provides the ChangeMessageVisibility API for this. MassTransit does not seem to support this native AWS interface. Which is still no problem by itself, as it boils down to calling a HTTP endpoint (or using the SQSClient SDK, C# in our case).
For this to work one needs the RequestHandle. It is part of the raw response when the AWS CLI is used to receive the message (aws sqs receive-message --queue-url http://localhost:4566/queue/my-qeueu).
{
"MessageId": "878eb3d8-2d1e-406f-87b6-24a3d493f5a9",
"ReceiptHandle": "878eb3d8-2d1e-406f-87b6-24a3d493f5a9#464f467d-82a9-421f-bc10-cddd63630b7d",
...left out for brevity
}
Now the message ID is known at run time when inspecting the ConsumeContext<TMyMessage>.ReceiveContext.TransportHeaders['MessageId']. The RequestHandle, however, is nowhere to be found, or so it seems.
So how can one change the visibility timeout of the message in a AWS SQS queue, consumed by a MassTransit consumer?
As this is a duplicate of the same question asked on GitHub Discussions, the same answer would be that MassTransit automatically extends the visibility timeout until the consumer completes.
Up to the maximum of 12 hours that SQS allows.
I'm new to Amazon SQS and I'm trying to implement it in one of my C# applications. From what I've seen so far from all the tutorials and blogs is that you keep polling the queue for messages and then if there are messages you loop through the queue and read each message and then process it.
I was thinking if there is any callback mechanism that is available. So that whenever a new message arrives in the queue the callback function is called?
Appreciate your help with this and thanks in advance.
We have created Azure service bus queue and have a azure function - servicebus queue trigger subscribed to it. We now want this queue to convert to Topic as we will need multiple subscription to it. is there a way that we can convert queue to Topic without deleting queue and re creating this as a Topic. I understand that Azure function code which is in c# needs to be changed so it points to service bus topic.
Thanks,
Punit Singhi
Use the ForwardTo feature for forwarding a message to the topic entity.
I've been working on sending and receiving ActiveMQ messages between .net applications and everything work great when there is a active Consumer Connection.
However there are scenarios that I won't be able to have a active Consumer, example: web application consuming active messages.
I notice that any message not delivered to a consumer is place in a "Messages Enqueued" bucket.
Is there a way I can consume an already send and Enqueued message for a particular topic?
I am workign with Topic only, as shown on the example below
any tips from .net or even java might help me out thanks
Topics (except for durable topics) are inherently going to throw messages away if you don't have a consumer connected at the time the message is sent. That's just how topics work. My guess is that the web UI's column header is misleading- it probably should say "messages sent". You cannot retrieve them.
More on durable topics and queues here: http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
I'm setting up a project in ASP.NET C# to manage the bounces and complaints notification when I send massive emails with Amazon SES.
I've read around that the best way, for large amount of messages, is SQS and not the simpler SNS on HTTP endpoint.
I've found some ready code by Amazon Team:
https://sesblog.amazon.com/post/TxJE1JNZ6T9JXK/Handling-Bounces-and-Complaints
I've understood that SQS service must be called by me, is not pushing like SNS, but I don't understand how I have to call it, which URL and how to build the request.
Sign in to the AWS Management Console and open the Amazon SQS console at https://console.aws.amazon.com/sqs/.
Create a Queue. http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/CreatingQueue.html
Select the queue to which you want to subscribe an Amazon SNS topic.
Select Subscribe Queue to SNS Topic from the Queue Actions drop-down list.
From the Choose a Topic drop-down list, select an Amazon SNS topic to subscribe the queue to and then click Subscribe.
In the Topic Subscription Result dialog box, click OK.
You can verify the results of the topic's queue subscription by publishing to the topic and viewing the message that the topic sends to the queue. For detailed steps, see Test it out by publishing a message to the topic and reading the message from the queue.