I am working on a console application program that receives messages from an Azure IoT Device and is using a timer to receive the messages every two seconds. Upon receiving a message the device sends out a call to complete it. The issue that I am experiencing is that the message does not complete in Azure before the device receives it again, which results in the message getting reprocessed. I have tried to filter incoming messages when it is the same message coming in multiple times, but the messages are coming in with the same message id whether or not they are duplicate messages or new messages. I do not have access to control the incoming message's message id field and make it unique, but that would solve the problem. The sequence number is unique for every message that comes in whether it is duplicate or not so I cannot use that as a filter either. Is there a way to filter a message to see if it is a duplicate without the message id field?
//Within Program.cs > Main():
_timer = new Timer(Operations, null, 0, _timerInterval); //_timerInterval is set to 2000
//Within Initialize class used to setup device client:
//Fully qualified namespace for DeviceClient:
//Microsoft.Azure.Devices.Client.DeviceClient
string connectionString = "code removed for example";
var deviceClient = DeviceClient.CreateFromConnectionString(connectionString);
//Within Operations class:
var message = await deviceClient.ReceiveAsync();
if (message != null && !string.IsNullOrEmpty(message?.MessageId))
{
//Filtering message based on MessageId
if (_memoryCache.Get(message.MessageId) == null)
{
_memoryCache.Set(message.MessageId, message.MessageId, DateTimeOffset.UtcNow.AddMinutes(10));
await deviceClient.CompleteAsync(message);
//Processing message
await ProcessMessage(message);
}
else
{
await deviceClient.RejectAsync(message);
}
}
You can use Microsoft.Azure.Devices.Client.Message package to retrieve the Device client message values.
Use the IOT explicit unique id in message to check the duplicates while receiving.
Follow the below code to check the duplicate value
List<string> FinalResponse = new List<string>();
Microsoft.Azure.Devices.Client.Message Response = await deviceClient.ReceiveAsync(TimeSpan.FromSeconds(10));
if (Response == null)
{
await Task.Delay(10).ConfigureAwait(false);
continue;
}
//here you can use the explicit properties like message id or correlation Id
Trace.WriteLine(Response.MessageId.ToString());
await this.deviceClient.CompleteAsync(Response);
var content = Encoding.UTF8.GetString(Response.GetBytes());
FinalResponse.Add(content);
Either you can use above one or use below conditions
Create a List and ADD all values to the list which get from Device
Add condition to ignore If any duplicates occur while insert into list.
Then send unduplicated values to Azure.
Related
I am using Azure.Messaging.ServiceBus nuget package to work with Azure service bus. We have created a topic and a subscription. The subscription has 100+ messages. We want to read all the message and continue to read message as they arrive.
Microsoft.Azure.ServiceBus package (deprecated now) provided RegisterMessageHandler which use to process every incoming message. I am not able to find similar option under Azure.Messaging.ServiceBus nuget package.
I am able to read one message at a time but I have to call await receiver.ReceiveMessageAsync(); every time manually.
To receive multiple messages (a batch), you should use ServiceBusReceiver.ReceiveMessagesAsync() (not plural, not singular 'message'). This method will return whatever number of messages it can send back. To ensure you retrieve all 100+ messages, you'll need to loop until no messages are available.
If you'd like to use a processor, that's also available in the new SDK. See my answer to a similar question here.
As suggested by #gaurav Mantri, I used ServiceBusProcessor class to implement event based model for processing messages
public async Task ReceiveAll()
{
string connectionString = "Endpoint=sb://sb-test-today.servicebus.windows.net/;SharedAccessKeyName=manage;SharedAccessKey=8e+6SWp3skB3Aedsadsadasdwz5DU=;";
string topicName = "topicone";
string subscriptionName = "subone";
await using var client = new ServiceBusClient(connectionString, new ServiceBusClientOptions
{
TransportType = ServiceBusTransportType.AmqpWebSockets
});
var options = new ServiceBusProcessorOptions
{
// By default or when AutoCompleteMessages is set to true, the processor will complete the message after executing the message handler
// Set AutoCompleteMessages to false to [settle messages](https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-transfers-locks-settlement#peeklock) on your own.
// In both cases, if the message handler throws an exception without settling the message, the processor will abandon the message.
AutoCompleteMessages = false,
// I can also allow for multi-threading
MaxConcurrentCalls = 1
};
await using ServiceBusProcessor processor = client.CreateProcessor(topicName, subscriptionName, options);
processor.ProcessMessageAsync += MessageHandler;
processor.ProcessErrorAsync += ErrorHandler;
await processor.StartProcessingAsync();
Console.ReadKey();
}
public async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
Console.WriteLine(body);
// we can evaluate application logic and use that to determine how to settle the message.
await args.CompleteMessageAsync(args.Message);
}
public Task ErrorHandler(ProcessErrorEventArgs args)
{
// the error source tells me at what point in the processing an error occurred
Console.WriteLine(args.ErrorSource);
// the fully qualified namespace is available
Console.WriteLine(args.FullyQualifiedNamespace);
// as well as the entity path
Console.WriteLine(args.EntityPath);
Console.WriteLine(args.Exception.ToString());
return Task.CompletedTask;
}
I have a conversation with two participants:
var createdConversation = await ConversationResource.CreateAsync(uniqueName: uniqueName);
var sender = await ParticipantResource.CreateAsync(pathConversationSid: createdConversation.Sid, identity: "sender");
var receiver = await ParticipantResource.CreateAsync(pathConversationSid: createdConversation.Sid, identity: "receiver");
And I want to send a initial message in this newly created conversation:
var message = await MessageResource.CreateAsync(
pathConversationSid: createdConversation.Sid,
author: sender.Identity,
body: intitialMessage);
After that, I want this new message to be read for the sender and unread for receiver which seems not possible.
I can set the lastReadMessageIndex of the sender to the created message.Index but this won't create read horizon for the receiver. And as long as the lastReadMessageIndex of the receiver is null, the unreadMessagesCount will be also null, and it will look like the receiver has no new messages in this conversation.
Any suggestions on how I can create a new conversation with initial message, which will be displayed as unread for one of the users?
Currently it is not possible to do this. This has been raised internally at Twilio as a feature gap.
In the meantime you may need to workaround it. One thing I thought of would be to mark any conversations that have a lastReadMessageIndex of null as unread in your UI.
As there is no "easy way" to achieve this, I am loading the conversation and its messages to calculate the unreadMessagesCount.
var conversation = await UserConversationResource.FetchAsync(
pathUserSid: userSid,
pathConversationSid: conversationSid);
var conversationHasNoReadHorizon = conversation.UnreadMessagesCount == null;
if (conversationHasNoReadHorizon)
{
var messages = await MessageResource.ReadAsync(pathConversationSid: conversationSid);
return messages.Count(message => message.Author != callerId);
}
else
{
return conversation.UnreadMessagesCount ?? 0;
}
I'm using .Net core to access an SQS queue and get a message from that queue.
Everything works fine.
For inserting a message :
var sendMessageRequest = new SendMessageRequest();
sendMessageRequest.QueueUrl = queueName;
sendMessageRequest.MessageBody = message;
sqs.SendMessage(sendMessageRequest);
Notice that I can only insert a single message :
However when I was looking at the ReceieveMessage method :
public static void GetMessage(AmazonSQSClient sqs , string queueName )
{
var receiveMessageRequest = new ReceiveMessageRequest { QueueUrl = queueName };
var receiveMessageResponse = sqs.ReceiveMessage(receiveMessageRequest);
I've noticed that I can only get a list :
Question:
I don't understand how come I can insert a single message , but read a list of messages ?
Even if I have many messages in my queue , this code ^ only fetches one message :
Where I have 6 messages :
If you want to send multiple messages in a single API call, there is SendMessageBatch documented here and here.
If you want to receive only a single message per request, you would set ReceiveMessageRequest.MaxNumberOfMessages to 1.
My problem is this; I'm making a bot with Discord.NET 1.0.2 that when a user sends a message, check if there is a file attached to that message and then remove it.
Now I made it, but there is one issue with that. The message won't be deleted until the file was uploaded. I have tried various ways but I can't come up with a solution to how I can wait until the file is uploaded, and then delete the message.
Any help is greatly appreciated.
This is what I'm working with:
private async Task CheckForImageMessage(SocketMessage s)
{
var msg = s as SocketUserMessage;
if (msg == null) return;
var context = new SocketCommandContext(_client, msg);
if (context.Message.Attachments.ElementAt(0) != null)
{
}
}
}
First of all, you are not making a command, so there is very little reason for you to be using a command context at all - message is already accessible to you as s (Attachments collection exists under the IMessage interface).
Second, you can't possibly intercept messages before the file is uploaded. If I understood correctly, you wish to delete any messages with attachments included? If so, an Any method on the Attachments collection would do.
private async Task RemoveAttachmentMsgsAsync(SocketMessage msg)
{
// check if the source channel is a message channel in a guild
if (msg.Channel is SocketTextChannel textChannel)
{
// get the current user to check for permissions
var currentUser = textChannel.Guild.CurrentUser;
// bail if the bot does not have manage message permission
if (!currentUser.GetPermissions(textChannel).ManageMessages) return;
}
// if we made it this far, we can assume that the bot has permission for
// the channel (including DM channel)
// use LINQ Any to check if the attachment contains anything
if (msg.Attachments.Any())
await msg.DeleteAsync();
}
I have a Use Case where I need to queue a select number of messages when the current queue length drops below a specified value. Since I'm running in Azure, I'm trying to use the RetrieveApproximateMessageCount() method to get the current message count. Everytime I call this I get an exception stating StorageClientException: The specified queue does not exist.. Here is a review of what I've done:
Created the queue in the portal and have successfully queued messages to it.
Created the storage account in the portal and it is in the Created/Online state
Coded the query as follows (using http and https options):
var storageAccount = new CloudStorageAccount(
new StorageCredentialsAccountAndKey(_messagingConfiguration.StorageName.ToLower(),
_messagingConfiguration.StorageKey), false);
var queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference(queueName.ToLower());
int messageCount;
try
{
messageCount = queue.RetrieveApproximateMessageCount();
}
catch (Exception)
{
//Booom!!!!! in every case
}
// ApproximateMessageCount is always null
messageCount = queue.ApproximateMessageCount == null ? 0 : queue.ApproximateMessageCount.Value;
I've confirmed the name is cased correctly with not special characters, numbers, or spaces and the resulting queue Url appears as though its correct formed based on the API documentations (e.g. http://myaccount.queue.core.windows.net/myqueue)
Can anyone help shed some light on what I'm doing wrong.
EDIT
I've confirmed that using the MessageFactory I can create a QueueClient and then enqueue/dequeue messages successfully. When I use the CloudStorageAccount the queue is never present so the counts and GetMessage routines never work. I am guessing these are not the same thing??? Assuming, I'm correct, what I need is to measure the length of the Service Bus Queue. Is that possible?
RetrieveApproximateMessageCount() has been deprecated
if you want to use ApproximateMessageCount to get result try this
CloudQueue q = queueClient.GetQueueReference(QUEUE_NAME);
q.FetchAttributes();
qCnt = q.ApproximateMessageCount;
The CloudQueue method has been deprecated (along with the v11 SDK).
The following snippet is the current replacement (from the Azure Docs)
//-----------------------------------------------------
// Get the approximate number of messages in the queue
//-----------------------------------------------------
public void GetQueueLength(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];
// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);
if (queueClient.Exists())
{
QueueProperties properties = queueClient.GetProperties();
// Retrieve the cached approximate message count.
int cachedMessagesCount = properties.ApproximateMessagesCount;
// Display number of messages.
Console.WriteLine($"Number of messages in queue: {cachedMessagesCount}");
}
}
https://learn.microsoft.com/en-us/azure/storage/queues/storage-dotnet-how-to-use-queues?tabs=dotnet#get-the-queue-length