At the moment I have a C# service that is reading messages off the queue (Websphere MQ) and writing them in a database.
Everytime I do a GET the message dissappears from queue. I would like an additional functionality though. I prefer to read a message off the queue and remove it in from the queue only after the write in the database was succesful. Please note I do all these in a multithreaded application. I know there is a way to browse the queue but this doesn't really provide the functionality I need.
I'm writing my firts WMQ application, and I know I'll run into this issue very soon, so I found your question.
I've found this http://www.mqseries.net/phpBB2/viewtopic.php?t=43043&sid=11ad2d587dbd19056836ccc3f8943e5f (specifing MQOO_BROWSE option while opening the queue) in other forum, I haven't tried it yet, but it think it worth a try...
[]'s
I have implemented similar functionality in C++. Hopefully this helps you or someone.
You can browse messages without removing them from the queue using options MQGMO_BROWSE_FIRST and MQGMO_BROWSE_NEXT.
How do I browse a Websphere MQ message without removing it?
Store message identifiers in a list or in any other suitable data structure.
Write messages to the database.
Then get messages from the queue normally without BROWSE option. ImqQueue::Get takes two parameters: options and ImqMessage. Set message identifier to ImqMessage-class before calling get. ImqMessage acts as a filter. You can select
only those messages that have been written successfully to the database.
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/index.jsp?topic=%2Fcom.ibm.mq.amqzan.doc%2Fuc10330_.htm
Related
I started to write my own bot for a discord server using Discord.net. I already made the first command (which still doesn't work correctly, but that's for another story), and now that I am testing, it has a chat full of commands I sent. My goal is to instantly delete those messages as I send them (let's say I send .command something, and this should appear in a chat just for a little moment). I tried to look it up in documentation and some tutorials but couldn't find anything. I have only basic knowledge of programming, so I would be happy if you could tell me how to achieve this and explain how it works.
To answer the question of how to delete a message, you use the Context.Channel.DeleteMessageAsync function. It takes a message id as the parameter, so you will need to have the message data in memory at the time.
To answer the question of how to delete a message that you have just sent, you can use the above delete function alongside the Context.Message data which gives you the message data of the message that triggered the command. It will look like the following:
[Command("TestDelete")]
public async TestDeleteCommand()
{
await Context.Channel.DeleteMessageAsync(Context.Message.Id);
}
You can find more information of the DeleteMessageAsync function by looking at the Discord.Net docs
We're using ActiveMQ locally to transfer data between 5 processes that turn simultaneously.
I have some data I need to send to a process, both at runtime (which works perfectly fine), but also a default value on start. Thing is it is published when the process starts, it just doesn't read because it wasn't subscribed to the topic at the time the data was sent.
I have multiple solutions : I could delay the first publishing for a moment so that the process has time to launch (which doesn't seem very appealing) ; or is there a way to send all stored previously non-treated messages to some process that just subscribed ?
I'm coding in C#.
I don't have any experience with ActiveMQ, but other message system usually have an option which marks the subscription as persistent, which means that; after the first subscription; the message queue itself checks if a certain message is delivered to that system and retries with a timeout. In this scenario you need to start the receiver at least 1 time.
If this is not an option and you want to plug in receiver afterwards, you might want to consider a setup of your messages which allows you to retrieve the full state, i.e. if you send total-messages instead of differential- messages.
After a little google, I came upon this definition durable subscribers, I hope this helps:
See:
http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
and
http://activemq.apache.org/manage-durable-subscribers.html
since you are using C# client i don't konw if this is supported
topic = new ActiveMQTopic("TEST.Topic?consumer.retroactive=true");
http://activemq.apache.org/retroactive-consumer.html
So, another solution is to configure this behavior on the broker side by adding that to the activemq.xml and restart :
The subscription recovery policy allows you to go back in time when
you subscribe to a topic.
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<subscriptionRecoveryPolicy>
<timedSubscriptionRecoveryPolicy recoverDuration="10000" />
<fixedCountSubscriptionRecoveryPolicy maximumSize="10000" />
</subscriptionRecoveryPolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
http://activemq.apache.org/subscription-recovery-policy.html
I went around the issue by sending a message from each process when they're launched back to the main one, and then only sending the info I needed to send.
Introduction
We exchange income data with an external party. Each year income tax regulations change and a new message schema has to be implemented. Altogether we now have 8 different schema versions each of which are deployed in a separate 'year income tax' application and this amount increases by 1 each year.
Because we pay our hosting company per installed application, we want to decrease the amount of applications installed.
All these applications are functionally equal, which means we validate incoming messages, and forward valid messages into a specific MQSeries queue. Each invalid message is routed to a response queue. Each application has it's own 'valid' and 'invalid' message queues.
The plan
One generic application that processes all 8(+) messages. New schemas must be deployable without application changes or downtime for previous, running 'income year tax' flows.
So far...
I can receive multiple messages on the same BizTalk receive port (MessageType XmlDocument) and am able to validate these messages dynamically in an orchestration by calling a custom receive pipeline (XML Disassembler + XML Validator). Exceptions as well as valid messages are processed as prescribed. There are no references between the Schemas and the generic application, so schemas can be deployed without need to stop running processes. So far, so good.
The orchestration has 1 receive shape, and 2 send shapes (valid, invalid).
SSO contains the values for routing the 'valid' and 'invalid' messages to their correct queue. Based on the incoming messagetype SSO is questioned for the correct 'valid' or 'invalid' queuedefinition.
The problem
I have previously dealt with dynamic FTP, FILE, WCF and SMTP ports, which all worked flawlessly after supplying the adapter with the correct Context Properties. Even MSMQ seems to have a fairly straightforward approach on dynamically setting transport properties.
However, I cannot seem to find MQSeries MQMT ContextProperties to set the queuedefinition dynamically.
Microsoft does not provide much information on this, and extensive searches on the internet hasn't provided me with anything useful (examples) either.
I tried matching IBM's docs with Microsoft's, but altogether I am now stuck.
I would suggest to use MQSC adapter for IBM MQ integration. It is part of Host Integration Server MSI. It only requires MQ client to be installed on the server Vs MQ Server for Windows installation required by MQSeries adapter.
Set the OutboundTransportLocation property in following format mqsc://{channelName}/tcp/{server{({port})/{queuemanager}/{queuename}
TransportType = MQSC
Context Properties - Schema can be found within assembly MQSeriesEx.MQSPropertySchemaEx with namespace (http://schemas.microsoft.com/BizTalk/2003/mqs-properties).
There are only few context properties you would need to set if at all required.
Channel_HeartBeat
Channel_MaxMessageLength
Channel_UserId
Channel_Password
ConnectionTimeout
If additional properties are required than use MQSeries.MQSPropertySchema context properties.
Thanks Vikas for your suggestion.
I followed your directions and found it works!
However, I found it a little more complicated than needed as it required me configuring channel names for each flow.
The solution that best suited me was the one I had in mind all along, and it was right before me. My attempts failed because I made a fatal mistake by setting the outgoing message's properties where I should have set the dynamic send port's properties.
SendPort(Microsoft.XLANGs.BaseTypes.Address)="MQS://SERVER/QMANAGER/QUEUENAME";
I have a redis instance that publishes messages via different topics. Instead of implementing a complex heartbeat mechanism (complex because the instance would stop publishing messages after some time if they are not consumed), is there a way to check whether pubs are consumed by anyone?
For example, instance RedisServer publishes messages to topic1 and topic2. RedisClient1 subscribes to topic1 and RedisClient2 subscribes to topic2. When RedisClient2 for whatever reason stops consuming messages of topic2 then I want RedisServer to know about it and decide when to stop publishing messages to topic2. The discontinuation of topic2 consumption is unpredictable hence I am not able to inform RedisServer of the discontinuation/unsubscription.
I thought if there was a way for a redis instance to know whether messages of a certain topic are consumed or not then that would be very helpful information.
Any idea whether that is possible?
Given you are using a recent-enough version of redis (> 2.8.0) these two commands may help you:
PUBSUB CHANNELS [pattern]
Which lists the currently active channels ( = channel having at least one subscriber) matching the pattern.
PUBSUB NUMSUB [chan1 ... chanN]
Which returns the number of subscribers for the specified channels (doesn't work for patterns however).
Note: Both solutions won't enable you to determine if a message was truely processed! If you need to know about completion of tasks (if your messages are triggering something), then I would recommend searching for a full blown job queue (for example Resque, if you want to stick with Redis)
Edit: Here's the Redis doc. for all of the above: http://redis.io/commands/pubsub
You can also use the result of PUBLISH. It will give you the number of subscribers that received the message: http://redis.io/commands/publish
This way you don't need to poll the PUBSUB command, just do your "stop publishing" messages logic after you publish a message.
At most you publish one message with no one subscribing.
I'm sending a message to a private queue via c# :
MessageQueue msgQ = new MessageQueue(#".\private$\aaa");
msgQ.Formatter = new XmlMessageFormatter(new[] { typeof (String) });
msgQ.Send(msg);
It does work and I do see the message in the queue.
However, is there any way to get an ACK whether the message got to the queue with success ?
ps
BeginPeek and PeekCompleted is an event which is raised when a message becomes available in the queue or when the specified interval of time has expired. it is not helping me because I need to know if the message that I sent was received by msmq. BeginPeek will be raised also if someone else entered a message to the queue. and the last thing I want is to check via BeginPeek - from who this message comes from.
How can I do that?
ps2
Or maybe I don't have to worry since msgQ.Send(msg); will raise an exception if a message wasn't inserted....?
I think what you are trying to do should not be handled in code. When you send the message, it is placed in the outgoing queue. There are numerous reasons why it would not reach the destination, such as a network partition or the destination queue being full. But this should not matter to your application - as far as it is concerned, it sent the message, it committed transaction, it received no error. It is a responsibility of the underlying infrastructure to do the rest, and that infrastructure should be monitored to make sure there are no technical issues.
Now what should really be important to your application is the delivery guarantees. I assume from the scenario that you are describing that you need durable transactional queues to ensure that the message is not lost. More about the options available can be read here
Also, if you need some identifier to display to the user as a confirmation, a common practice is to generate it in the sending code and place it in the message itself. Then the handling code would use the id to do the required work.
Using transactional queues and having all your machines enroll in DTC transactions likely would provide what you're looking for. However, it's kinda a pain in the butt and DTC has side effects - like all transactions are enrolled together, including DB transactions.
Perhaps a better solution would to be use a framework like MassTransit or NServiceBus and do a request-response, allowing the reviecer to respond with actual confirmation message say not only "this has been delivered" but also "I acknowledge this" with timeout options.
As Oleksii have explained about reliable delivery.
However this can effect on performance.
What I can suggest is:
Why not create a MSMQ server on the machine that is sending MSG to other system.
What I am thinking is
Server 1 sends MSMSQ to Server 2
Server 2 receives adds to queue
Server 2 process queue/fire your code here to send a MSMQ msg to Server 1
Server 1 receives MSG (any successful msg with MSGId)
Do your further task
This approach can be an extra mile, but will keep your servers out of performance Lag.