Develop Reference

Apache NMS IBytesMessage has a prefix after dequeuing

I've tried my best to figure this out, but I'm guessing my inexperience with Dotnet/C# and message queues is showing through.
The problem
In a nutshell the situation I'm dealing with is, I have two projects.
The first project is a legacy Dotnet project using the Apache NMS library to consume messages from an ActiveMQ queue.
The queue is normally connected in production via SSL, but developing locally I have connected via TCP which all works fine and actively connects & dequeues as expected.
Once the messages are dequeued they are deserialised using the JsonConvert DeserializeObject method.
The second project is a NodeJS project which I've created to put messages onto the queue.
The messages are just a simple object, stringified to JSON.
I'm using the AMQP Rhea npm package to connect and send the messages, in AMQP format.
This again all works fine and the messages are enqueued as expected.
The problem is when I come to deserialising the message pulled from the queue.
There's an exception because it encounters a prefix to my JSON message - I have no idea where the prefix comes from.
I have a creeping suspicion it's because there's a clash between the protocols I'm using and that's causing the issue, but before I find myself in that particular rabbit hole - or just resorting to using a shaky method of removing the prefix - I'm hoping one of you with more experience can help point me in the right direction.
Code
Here's a couple of snippets of the offending C#:
This is how I'm dequeueing the messages:
string dequeuedMessage = string.Empty;
List<string> messages = new List<string>();
IMessage message = _consumer.Receive(waitForMessage);
if (message is ITextMessage || message is IBytesMessage)
{
dequeuedMessage = (message is ITextMessage) ? (message as ITextMessage).Text : Encoding.UTF8.GetString((message as IBytesMessage).Content);
messages.Add(dequeuedMessage);
}
else
break;
What's interesting is the message always comes through as type IBytesMessage, not what the previous author was expecting (ITextMessage), which is why I'm having that creeping suspicion.
And this is how I'm deserialising the message:
private IMessage BuildMessageObject(string message)
{
IMessage convertedMessage = JsonConvert.DeserializeObject<QueueMessage>(message);
return convertedMessage;
}
Here's a look at the prefix:
\0SpE\0SsE\0Sw\xb1\0\0\x01\xac{"Message":"Message Content Snipped for Brevity"}
It appears to always be the same, so maybe it means something to someone.
What I've Tried & Expectations
To quickly debug, I tried connecting to the queue in the Dotnet project using AMQP, but it appears that the Apache NMS package doesn't support this.
I'm not in a position to completely rewrite the connection with another package either.
I've also written something to dequeue the message in my Node app and I get the JSON back without the prefix, but this connects with the AMQP protocol which may be why?
Thanks for any help.
Edit: I should add that I'm using ActiveMQ Classic and not Artemis.

Answering my own question, it appears it is an issue with the amqp/rhea package and using the AMQP protocol as I suspected.
The problem, without running into details I can't fully explain, is that the message from the queue has been encoded and the message body includes that encoding.
The NMS library uses ActiveMQ's native OpenWire protocol.
I'm probably going to have to resolve this by moving from using the amqp/rhea package and trying to find an openwire equivalent - which is preferable than trying to find an AMQP equivalent for the Dotnet project.
Edit:
As it turns out, the rhea package is sufficient to solve the problem.
In sending the message body, I've instead created it as a Buffer instead of type Message:
const messageAsBuffer = Buffer.from(message);
sender.send(messageAsBuffer, undefined, 1);
where undefined can be a message tag and 1 can be any integer that is not 0, specifying the message is not in AMQP format.

IBMMQDotnetClient vs IBMXMSDotnetClient

I am working on a project, which needs to connect to IBM MQ using c#, and considering which NuGet package is the best one.
However, there are 2 NuGet packages IBMMQDotnetClient and IBMXMSDotnetClient and both of them are provided by the official (IBM).
After reading a few references,
Difference between nuget packages IBMMQDotnetClient and IBMXMSDotnetClient (IBM MQ support for .Net Core)
https://www.ibm.com/support/pages/xms-net-%E2%80%93-overview
It looks like that me that there are 2 main differences between these 2 API.
IBMXMSDotnetClient provides more functionalities
IBMXMSDotnetClient provides JMS-like API. If someone is using Java before, it is more easier to port the Java code to .Net
Another difference I noticed is that the way they are connecting to the MQ server (QueueManager vs Connection and Session). But it is not really a problem, just different styles as far as I can see.
IBMMQDotnetClient
MQEnvironment.Hostname = hostName;
MQEnvironment.Channel = channel;
// Queue Manager object
MQQueueManager queueManager = new MQQueueManager(queueManagerName);
int openOptions = MQC.MQOO_INPUT_AS_Q_DEF | MQC.MQOO_OUTPUT;
MQQueue system_default_local_queue = queueManager .AccessQueue(queueName, openOptions);
IBMXMSDotnetClient
XMSFactoryFactory factoryFactory;
IConnectionFactory cf;
IDestination destination;
factoryFactory = XMSFactoryFactory.GetInstance(XMSC.CT_WMQ);
cf = factoryFactory.CreateConnectionFactory();
cf.SetStringProperty(XMSC.WMQ_HOST_NAME, hostName);
cf.SetIntProperty(XMSC.WMQ_PORT, port);
cf.SetStringProperty(XMSC.WMQ_CHANNEL, channel);
cf.SetIntProperty(XMSC.WMQ_CONNECTION_MODE, XMSC.WMQ_CM_CLIENT);
// Queue Manager just as an simple string property here
cf.SetStringProperty(XMSC.WMQ_QUEUE_MANAGER, queueManagerName);
If it is the case, I cannot see any reason why I do not choose IBMXMSDotnetClient.
But the weird thing is that there are 327K downloads for IBMMQDotnetClient while there are only 160k downloads for IBMXMSDotnetClient.
Or there are some key advantages of IBMMQDotnetClient I missed?

IBMMQDotNet provides MQ native APIs in .NET language while IBMXMSDotNet provides JMS style of APIs in .NET. However there is one major difference between the two APIs: IBMXMSDotNet provides asynchronous message consumer while IBMMQDotNet does not. Asynchronous message consumption is a type of consuming messages where IBMXMSDotNet automatically calls application registered callback method whenever a message arrives in MQ queue. With IBMMQDotNet, you will need to do a synchronous Get call (with or without timeout) to receive messages.
Hope this helps
Update: 16th May 2022
One example: In case of MQDotNet, you can change selection criteria to read messages matching another criteria without reopening a queue. For example you can say "select message matching groupId x" first from the queue to read messages matching the given groupId. Then you can say "select message matching groupId y" from the same queue without reopening queue. In case of XMSDotNet, the selection createria is specified while creating the consumer (which internally opens a MQ queue). Consumer must be recreated if you want to change selection criteria to consume messages with another criteria

Amazon WorkMail timeout .net core SmtpClient

I am trying to send a standard email message through AWS WorkMail using the SmtpClient in .NET Core. The configuration is very standard according to Amazon Documentation:
https://docs.amazonaws.cn/en_us/general/latest/gr/workmail.html
"Smtp": {
"MailServer": "smtp.mail.eu-west-1.awsapps.com",
"MailPort": "465",
"SenderName": "Us us us",
"FromAddress": "email#domain.com",
"Username": "email#domain.com",
"Password": "Password1$",
"EnableSsl": "true"
},
The settings are injected and the SmtpClient gets properly instantiated and the sending of the email is just:
var mail = new MailMessage
{
From = new MailAddress(_smtpSettings.FromAddress, _smtpSettings.SenderName),
SubjectEncoding = Encoding.UTF8,
BodyEncoding = Encoding.UTF8,
IsBodyHtml = true,
Body = message,
Subject = subject,
Priority = MailPriority.High
};
mail.To.Add(new MailAddress(sendToEmail));
_smtpClient.Send(mail);
Sadly, the sending always fails with a gateway timeout. We tried switching for 587 for STARTTLS and providing the Username without the domain (#). The server sending has proper SSL certificate installed and the mailserver is exactly the one we have our smtp on. Although this would not be a relevant solution, I also increased the timeout to 10 seconds (even though this would not be a solution).
Any idea what could be wrong? I am tearing my hair already. Seems to be an issue with AWS WorkMail.

Just adding to the existing comment. SMTPClient is actually obsolete/deprecated but is not being marked as so. See comments here (https://github.com/dotnet/dotnet-api-docs/issues/2986#issuecomment-430805681)
Essentially it boils down to SmtpClient hasn't been updated in years and is missing many features. The .NET Core team wanted to mark it as obsolete, but some developers have existing projects with "Warnings as Errors" turned on. It would instantly make any project that is using SmtpClient with Warnings as Errors turned on suddenly stop building. So... It's kinda deprecated, but not being marked so in some official docs.
MailKit is actually being pushed by Microsoft full stop for people to use when it comes to Email. Much to some developers chargrin who don't want to use a third party library for such a "simple" and common feature. Just in my personal experience, I think Mailkit is great and super easy to use. A quick guide to getting up and running is here : https://dotnetcoretutorials.com/2017/11/02/using-mailkit-send-receive-email-asp-net-core/

So the problem is that the default .NET Core SmtpClient does not support StartSSL (implicit SSL) which is the only accepted option by WorkMail. You see, WorkMail allows only connections that start from SSL and the SmtpClient first starts from unencrypted and then switches over to encrypted if it cannot connect.
If you are trying this, you will not get this to work using standard SmtpClient and as usually the case with Microsoft, they don't recognize it as an issue. You can either try tunneling or better just use one of the available libraries. Sadly, most of them are paid, there is AIM but it doesn't work with .NET Core and I didn't want to spend time porting not my own library to .NET Standard so I ended up using Mailkit.
There are some issues with the library though, first, before sending you have to call Connect which takes host and port as parameter, that means you cannot just inject premade smtpclient as singleton and have to instantiate it within the place of usage. That's dope and no interface also makes it unmockable which might blow your integration tests. Moreover, you have to do an ugly line before calling send like so:
emailClient.AuthenticationMechanisms.Remove("XOAUTH2");
But at least it works.

Dynamically set Endpoint definition in BizTalk Send Port using MQSeries adapter

Introduction
We exchange income data with an external party. Each year income tax regulations change and a new message schema has to be implemented. Altogether we now have 8 different schema versions each of which are deployed in a separate 'year income tax' application and this amount increases by 1 each year.
Because we pay our hosting company per installed application, we want to decrease the amount of applications installed.
All these applications are functionally equal, which means we validate incoming messages, and forward valid messages into a specific MQSeries queue. Each invalid message is routed to a response queue. Each application has it's own 'valid' and 'invalid' message queues.
The plan
One generic application that processes all 8(+) messages. New schemas must be deployable without application changes or downtime for previous, running 'income year tax' flows.
So far...
I can receive multiple messages on the same BizTalk receive port (MessageType XmlDocument) and am able to validate these messages dynamically in an orchestration by calling a custom receive pipeline (XML Disassembler + XML Validator). Exceptions as well as valid messages are processed as prescribed. There are no references between the Schemas and the generic application, so schemas can be deployed without need to stop running processes. So far, so good.
The orchestration has 1 receive shape, and 2 send shapes (valid, invalid).
SSO contains the values for routing the 'valid' and 'invalid' messages to their correct queue. Based on the incoming messagetype SSO is questioned for the correct 'valid' or 'invalid' queuedefinition.
The problem
I have previously dealt with dynamic FTP, FILE, WCF and SMTP ports, which all worked flawlessly after supplying the adapter with the correct Context Properties. Even MSMQ seems to have a fairly straightforward approach on dynamically setting transport properties.
However, I cannot seem to find MQSeries MQMT ContextProperties to set the queuedefinition dynamically.
Microsoft does not provide much information on this, and extensive searches on the internet hasn't provided me with anything useful (examples) either.
I tried matching IBM's docs with Microsoft's, but altogether I am now stuck.

I would suggest to use MQSC adapter for IBM MQ integration. It is part of Host Integration Server MSI. It only requires MQ client to be installed on the server Vs MQ Server for Windows installation required by MQSeries adapter.
Set the OutboundTransportLocation property in following format mqsc://{channelName}/tcp/{server{({port})/{queuemanager}/{queuename}
TransportType = MQSC
Context Properties - Schema can be found within assembly MQSeriesEx.MQSPropertySchemaEx with namespace (http://schemas.microsoft.com/BizTalk/2003/mqs-properties).
There are only few context properties you would need to set if at all required.
Channel_HeartBeat
Channel_MaxMessageLength
Channel_UserId
Channel_Password
ConnectionTimeout
If additional properties are required than use MQSeries.MQSPropertySchema context properties.

Thanks Vikas for your suggestion.
I followed your directions and found it works!
However, I found it a little more complicated than needed as it required me configuring channel names for each flow.
The solution that best suited me was the one I had in mind all along, and it was right before me. My attempts failed because I made a fatal mistake by setting the outgoing message's properties where I should have set the dynamic send port's properties.
SendPort(Microsoft.XLANGs.BaseTypes.Address)="MQS://SERVER/QMANAGER/QUEUENAME";

Multiple ServiceStack applications with one RabbitMQ server

I have created 2 ServiceStack applications that run as Windows services via TopShelf and make use of one RabbitMQ server. Unfortunately when I start the second application the following exception occurs:
Exception in Rabbit MQ Server: The AMQP operation was interrupted: AMQP close-reason, initiated by Peer, code=406, text="PRECONDITION_FAILED - cannot redeclare exchange 'mx.servicestack.topic' in vhost '/' with different type, durable, internal or autodelete value"
The startup code contains the following code:
App 1
...
var rabbitMqServer = new RabbitMqServer();
rabbitMqServer.RegisterHandler<BusMessages.CrawlRequest>(
n =>
{
var request = n.GetBody();
this.Crawl(request);
return null;
});
rabbitMqServer.Start();
...
App 2
...
var rabbitMqServer = new RabbitMqServer();
rabbitMqServer.RegisterHandler<SendMailRequest>(
message =>
{
SendMail(message.GetBody());
return null;
});
rabbitMqServer.Start();
...
The problem seems to be with the exchange named mx.servicestack.topic, which is defaulted by ServiceStack. Does anyone know a solution to circumvent this or change the Exchange name so I can use multiple (rather default) ServiceStack applications in combination with the same RabbitMQ server?
Update
As I was looking into it more deeply it seemed to be a bug in ServiceStack.RabbitMq v4.0.31 (used in App 1). In that version the default exchange mx.servicestack.topic is added as a fanout exchange type instead of a topic exchange type. App 2 was using ServiceStack.RabbitMq v4.0.40 which tries to add/use the exchange mx.servicestack.topic as a topic exchange type, as it should be. Upgrading the ServiceStack packages to version 4.0.40 for App 1 fixed this issue.
I prefer the way of segregation for different applications like Alain explains in his answer https://stackoverflow.com/a/31209330/1278669.
However, for different applications working in the same (small) customers' domain it's very doable to use the default exchanges like ServiceStack creates.
Last but not least, I found a dirty workaround to get App 2 running next to App 1 without upgrading the ServiceStack packages of App 1. That's done by doing the following:
...
QueueNames.ExchangeTopic = "mx.App2.topic";
var rabbitMqServer = new RabbitMqServer();
...

You need multiple vhosts in the RabbitMQ server to segregate your ServiceStack applications.
Instead of amqp://localhost:5672 you can use amqp://localhost:5672/vhostname when configuring your RabbitMqServer as described here:
https://github.com/ServiceStack/ServiceStack/wiki/Rabbit-MQ
In a practical deployment the RabbitMQ server wouldn't be on localhost. I'm using that above as a short step from where you currently are using the built-in default which is amqp://localhost:5672 when invoking new RabbitMqServer().
Virtual hosts need to be added on the RabbitMQ server ahead of time and users need to be created for them separately. They are effectively separate AMQP servers with shared infrastructure.
You can add vhosts with rabbitmqctl as follows
rabbitmqctl add-vhost vhostname