I am aware of IPC and have used it in several projects. My new requirements include two simple C# applications that need to exchange short messages like "do this" "done that". This should be bidirectional.
This is going to run on old systems (Win Xp .Net 3.5) without being able to configure them. The end user will just run the application.
WCF is too complicated for this task
Sockets over TCP require firewall configuration and might be blocked from antiviruses
Microsoft Message Queuing fits perfectly but requires MSMQ to be installed on all clients
Named pipes can be used, but I would like something in higher level (I feel like using assembly with them)
Is there a simple way to exchange data between C# applications?
You've got a couple options:
wcf, which you say you've considered and rejected; may I inquire as to what made you reject it?
named pipes, as you mentioned; it's actually reasonably trivial to wrap two named pipes into a bidirectional messaging system (pro tip: make it disposable and clean up the pipes on dispose lest you lock them up)
queuing systems, but usually you see these in many-to-many or one-to-many channels
serialize a message queue (simple queue structure) and put it in a memory mapped file, then read/write from either side
esoteric: use windows messaging, i.e. pinvoke send/post message
Although you said TCP is out...
i've been very keen to use ZeroMQ but just haven't had an opportunity. It's multi platform with a c# wrapper and is designed exactly for this sort of application messaging. Check out ZeroMQ at http://www.zeromq.org/.
Some sample code from one of their sites:
// Set up a publisher.
var publisher = new ZmqPublishSocket {
Identity = Guid.NewGuid().ToByteArray(),
RecoverySeconds = 10
};
publisher.Bind( address: "tcp://127.0.0.1:9292" );
// Set up a subscriber.
var subscriber = new ZmqSubscribeSocket();
subscriber.Connect( address: "tcp://127.0.0.1:9292" );
subscriber.Subscribe( prefix: "" ); // subscribe to all messages
// Add a handler to the subscriber's OnReceive event
subscriber.OnReceive += () => {
String message;
subscriber.Receive( out message, nonblocking: true );
Console.WriteLine( message );
};
// Publish a message to all subscribers.
publisher.Send( "Hello world!" );
Related
I am working on a project, which needs to connect to IBM MQ using c#, and considering which NuGet package is the best one.
However, there are 2 NuGet packages IBMMQDotnetClient and IBMXMSDotnetClient and both of them are provided by the official (IBM).
After reading a few references,
Difference between nuget packages IBMMQDotnetClient and IBMXMSDotnetClient (IBM MQ support for .Net Core)
https://www.ibm.com/support/pages/xms-net-%E2%80%93-overview
It looks like that me that there are 2 main differences between these 2 API.
IBMXMSDotnetClient provides more functionalities
IBMXMSDotnetClient provides JMS-like API. If someone is using Java before, it is more easier to port the Java code to .Net
Another difference I noticed is that the way they are connecting to the MQ server (QueueManager vs Connection and Session). But it is not really a problem, just different styles as far as I can see.
IBMMQDotnetClient
MQEnvironment.Hostname = hostName;
MQEnvironment.Channel = channel;
// Queue Manager object
MQQueueManager queueManager = new MQQueueManager(queueManagerName);
int openOptions = MQC.MQOO_INPUT_AS_Q_DEF | MQC.MQOO_OUTPUT;
MQQueue system_default_local_queue = queueManager .AccessQueue(queueName, openOptions);
IBMXMSDotnetClient
XMSFactoryFactory factoryFactory;
IConnectionFactory cf;
IDestination destination;
factoryFactory = XMSFactoryFactory.GetInstance(XMSC.CT_WMQ);
cf = factoryFactory.CreateConnectionFactory();
cf.SetStringProperty(XMSC.WMQ_HOST_NAME, hostName);
cf.SetIntProperty(XMSC.WMQ_PORT, port);
cf.SetStringProperty(XMSC.WMQ_CHANNEL, channel);
cf.SetIntProperty(XMSC.WMQ_CONNECTION_MODE, XMSC.WMQ_CM_CLIENT);
// Queue Manager just as an simple string property here
cf.SetStringProperty(XMSC.WMQ_QUEUE_MANAGER, queueManagerName);
If it is the case, I cannot see any reason why I do not choose IBMXMSDotnetClient.
But the weird thing is that there are 327K downloads for IBMMQDotnetClient while there are only 160k downloads for IBMXMSDotnetClient.
Or there are some key advantages of IBMMQDotnetClient I missed?
IBMMQDotNet provides MQ native APIs in .NET language while IBMXMSDotNet provides JMS style of APIs in .NET. However there is one major difference between the two APIs: IBMXMSDotNet provides asynchronous message consumer while IBMMQDotNet does not. Asynchronous message consumption is a type of consuming messages where IBMXMSDotNet automatically calls application registered callback method whenever a message arrives in MQ queue. With IBMMQDotNet, you will need to do a synchronous Get call (with or without timeout) to receive messages.
Hope this helps
Update: 16th May 2022
One example: In case of MQDotNet, you can change selection criteria to read messages matching another criteria without reopening a queue. For example you can say "select message matching groupId x" first from the queue to read messages matching the given groupId. Then you can say "select message matching groupId y" from the same queue without reopening queue. In case of XMSDotNet, the selection createria is specified while creating the consumer (which internally opens a MQ queue). Consumer must be recreated if you want to change selection criteria to consume messages with another criteria
We're using ActiveMQ locally to transfer data between 5 processes that turn simultaneously.
I have some data I need to send to a process, both at runtime (which works perfectly fine), but also a default value on start. Thing is it is published when the process starts, it just doesn't read because it wasn't subscribed to the topic at the time the data was sent.
I have multiple solutions : I could delay the first publishing for a moment so that the process has time to launch (which doesn't seem very appealing) ; or is there a way to send all stored previously non-treated messages to some process that just subscribed ?
I'm coding in C#.
I don't have any experience with ActiveMQ, but other message system usually have an option which marks the subscription as persistent, which means that; after the first subscription; the message queue itself checks if a certain message is delivered to that system and retries with a timeout. In this scenario you need to start the receiver at least 1 time.
If this is not an option and you want to plug in receiver afterwards, you might want to consider a setup of your messages which allows you to retrieve the full state, i.e. if you send total-messages instead of differential- messages.
After a little google, I came upon this definition durable subscribers, I hope this helps:
See:
http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
and
http://activemq.apache.org/manage-durable-subscribers.html
since you are using C# client i don't konw if this is supported
topic = new ActiveMQTopic("TEST.Topic?consumer.retroactive=true");
http://activemq.apache.org/retroactive-consumer.html
So, another solution is to configure this behavior on the broker side by adding that to the activemq.xml and restart :
The subscription recovery policy allows you to go back in time when
you subscribe to a topic.
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<subscriptionRecoveryPolicy>
<timedSubscriptionRecoveryPolicy recoverDuration="10000" />
<fixedCountSubscriptionRecoveryPolicy maximumSize="10000" />
</subscriptionRecoveryPolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
http://activemq.apache.org/subscription-recovery-policy.html
I went around the issue by sending a message from each process when they're launched back to the main one, and then only sending the info I needed to send.
I have a redis instance that publishes messages via different topics. Instead of implementing a complex heartbeat mechanism (complex because the instance would stop publishing messages after some time if they are not consumed), is there a way to check whether pubs are consumed by anyone?
For example, instance RedisServer publishes messages to topic1 and topic2. RedisClient1 subscribes to topic1 and RedisClient2 subscribes to topic2. When RedisClient2 for whatever reason stops consuming messages of topic2 then I want RedisServer to know about it and decide when to stop publishing messages to topic2. The discontinuation of topic2 consumption is unpredictable hence I am not able to inform RedisServer of the discontinuation/unsubscription.
I thought if there was a way for a redis instance to know whether messages of a certain topic are consumed or not then that would be very helpful information.
Any idea whether that is possible?
Given you are using a recent-enough version of redis (> 2.8.0) these two commands may help you:
PUBSUB CHANNELS [pattern]
Which lists the currently active channels ( = channel having at least one subscriber) matching the pattern.
PUBSUB NUMSUB [chan1 ... chanN]
Which returns the number of subscribers for the specified channels (doesn't work for patterns however).
Note: Both solutions won't enable you to determine if a message was truely processed! If you need to know about completion of tasks (if your messages are triggering something), then I would recommend searching for a full blown job queue (for example Resque, if you want to stick with Redis)
Edit: Here's the Redis doc. for all of the above: http://redis.io/commands/pubsub
You can also use the result of PUBLISH. It will give you the number of subscribers that received the message: http://redis.io/commands/publish
This way you don't need to poll the PUBSUB command, just do your "stop publishing" messages logic after you publish a message.
At most you publish one message with no one subscribing.
I just started to learn ZeroMQ and want to build a distributed webcrawler as an example while learing.
My idea is to have a "server", written in PHP, which accepts a url where the crawling should start.
Workers (C# cli) will have to crawl that url, extract links, and push them back into a stack on the server. The server keeps sending urls in the stack to workers.
Perhaps a redis will keep track of all crawled urls, so we dont crawl sites multiple times and have the ability to extract statistics of the current process.
I would like to have the server to distribute tasks evenly, be aware of new/missing workers and redistribute urls when a worker doesnt respond.
Why PHP for the server: i'm just very comfortable with PHP, that is all. I dont want to make the example/testing project more complicated.
Why C# for the minions: because it runs on most windows machines. I can give the executable to various friends which can just execute it and help me test my project.
The crawling process and redis functionality are not part of my question.
My first approach was the PUSH/PULL pattern, which generally works for my scenario, but isnt aware of it's minions. I think i need a DEALER/ROUTER broker in the middle and have to handle the worker-awareness for myself.
I found this question but i'm not really sure if i understand the answer...
I'm asking for some hints how to impement the zmq stuff. Is the dealer approach correct? Is there any way to get an automatic worker-awareness? I think I need some resources/examples, or do you think that i just need to dig deeper in the zmq guide?
However, some hints towards the right direction would be great :)
Cheers
I'm building a job/task distributor that works the same as your crawler, in principal, at least. Here's a few things I've learned:
Define All Events
Communication between server and crawlers will be based on different things happening in your system, such as dispatching work from server to crawler, or a crawler sending a heartbeat message to the server. Define the system's event types; they are the use cases:
DISPATCH_WORK_TO_CRAWLER_EVENT
CRAWLER_NODE_STATUS_EVENT
...
Define a Message Standard
All communication between server and crawlers should be done using ZMsg's, so define a standard that organizes your frames, something like this:
Frame1: "Crawler v1.0" //this is a static header
Frame2: <event type> //ex: "CRAWLER_NODE_STATUS_EVENT"
Frame3: <content xml/json/binary> //content that applies to this event (if any)
Now you can create message validators to validate ZMsgs received between peers since you have a standard convention all messages must follow.
Server
Use a single ROUTER on the server for asynchrounous and bidirectional communication with the crawlers. Also, use a PUB socket for broadcasting heartbeat messages.
Don't block on the ROUTER socket, use a POLLER to loop every 5s or whatever, this allows the server to do other things periodically, like broadcast heartbeat events to the crawlers; something like this:
Socket rtr = .. //ZMQ.ROUTER
Socket pub = .. //ZMQ.PUB
ZMQ.Poller poller = new ZMQ.Poller(2)
poller.register( rtr, ZMQ.Poller.POLLIN)
poller.register( pub, ZMQ.Poller.POLLIN)
while (true) {
ZMsg msg = null
poller.poll(5000)
if( poller.pollin(0)){
//messages from crawlers
msg = ZMsg.recvMsg(rtr)
}
//send heartbeat messages
ZMsg hearbeatMsg = ...
//create message content here,
//publish to all crawlers
heartbeatMsg.send(pub)
}
To address your question about worker awareness, a simple and effective method uses a FIFO stack along with the heartbeat messages; something like this:
server maintains a simple FIFO stack in memory
server sends out heartbeats; crawlers respond with their node name; the ROUTER automatically puts the address of the node in the message as well (read up on message enveloping)
push 1 object onto the stack containing the node name and node address
when the server wants to dispatch work to a crawler, just pop the next object from the stack, create the message and address is properly (using the node address), and off it goes to that worker
dispatch more work to other crawlers the same way; when a crawler responds back to the server, just push another object with node name/address back on the stack; the other workers won't be available until they respond, so we don't bother them.
This is a simple but effective method of distributing work based on worker availability instead of blindly sending out work. Check lbbroker.php example, the concept is the same.
Crawler (Worker)
The worker should use a single DEALER socket along with a SUB. The DEALER is the main socket for async communication, and the SUB subscribes to heartbeat messages from the server. When the worker receives a heartbeat messages, it responds to the server on the DEALER socket.
Socket dlr = .. //ZMQ.DEALER
Socket sub = .. //ZMQ.SUB
ZMQ.Poller poller = new ZMQ.Poller(2)
poller.register( dlr, ZMQ.Poller.POLLIN)
poller.register( sub, ZMQ.Poller.POLLIN)
while (true) {
ZMsg msg = null
poller.poll(5000)
if( poller.pollin(0)){
//message from server
msg = ZMsg.recvMsg(dlr)
}
if( poller.pollin(1)){
//heartbeat message from server
msg = ZMsg.recvMsg(sub)
//reply back with status
ZMsg statusMsg = ...
statusMsg.send(dlr)
}
The rest you can figure out on your own. Work through the PHP examples, build stuff, break it, build more, it's the only way you'll learn!
Have fun, hope it helps!
My problem is that I want to have a server application (on a remote computer) to publish certain events to several client computers. The server and client communicate using .Net-Remoting so currently I am using remoted .Net-Events to get the functionality. But there is one drawback: when the server (the event publisher) comes offline and is restarted, the clients lose the connection since the remote object references become invalid.
I am looking into Loosely Coupled Events and Transient COM Subscriptions to solve this issue. I put together a small demo application with one publisher and two subscribers. It works beautifully on one computer.
I am using the COMAdmin-Libraries to create a transient subscription for the event subscribers. The code looks like this:
MyEventHandler handler = new MyEventHandler();
ICOMAdminCatalog catalog;
ICatalogCollection transientCollection;
ICatalogObject subscription;
catalog = (ICOMAdminCatalog)new COMAdminCatalog();
transientCollection = (ICatalogCollection)catalog.GetCollection("TransientSubscriptions");
subscription = (ICatalogObject)transientCollection.Add();
subscription.set_Value("Name", "SubTrans");
subscription.set_Value("SubscriberInterface", handler);
string eventClassString = "{B57E128F-DB28-451b-99D3-0F81DA487EDE}";
subscription.set_Value("EventCLSID", eventClassString);
string sinkString = "{9A616A06-4F8D-4fbc-B47F-482C24A04F35}";
subscription.set_Value("InterfaceID", sinkString);
subscription.set_Value("FilterCriteria", "");
subscription.set_Value("PublisherID", "");
transientCollection.SaveChanges();
handler.Event1 += OnEvent1;
handler.Event2 += OnEvent2;
My question now is: what do I have to change in the subscription to make this work over a network? Is it even possible?
What about MSMQ? It seems perfect for what you are trying to achieve? You can use a traditional publish/subscribe model or multicast the messages.
This might be a step too far, but have you considered using WCF and the callback element of WCF?
Callback effectively turns the what was client into a server. To be honest, I don't know a great deal about callback and have only experimented. Perhaps worth a 10 minute google though.
If your server comes offline every once and a while I cannot see how you can avoid to poll it to check that it is alive.
As you are talking about COM and remote computers, I suspect you'll have to do some DCOM security configuration.