I've received some C# code from a colleague for interacting with a cRIO device connected over ethernet. I'm trying to improve the code quality to make it a bit more comprehensible for future users, however I'm struggling a little bit to extract some relevant information from the API documentation. My main question is whether there would be problems caused in leaving a NetworkVariableManager in the Connected state?
Right now the code uses a class which looks something like
public class RIOVar<T>
{
public readonly string location;
public RIOVar(string location)
{
this.location = location;
}
public T Get()
{
using(NetworkVariableReader<T> reader = new NetworkVariableReader<T>(location) )
{
reader.Connect();
return reader.ReadData().GetValue()
}
}
public void Write(T value)
{
using(NetworkVariableWriter<T> writer = new NetworkVariableWriter<T>(location) )
{
writer.Connect();
writer.WriteValue(value);
}
}
}
The actual class does a lot more than this, but the part that actually communicates with the cRIO basically boils down to these two methods and the location data member.
What I'm wondering about is whether it would be better to instead have reader and writer as class members and Connect them in the constructor (at the point that they are constructed the connection should be posible), but what I don't know is if this would have some adverse effect on the way the computer and RIO communicate with each other (maybe a connected manager uses some resource or the program must maintain some sort of register...?) and therefore the approach here of having the manager connected only for the read/write operation is better design.
Keeping a variable connected keeps its backing resources in memory:
threads
sockets
data buffers
These resources are listed in the online help, but it's unclear to me if that list is complete:
NationalInstruments.NetworkVariable uses multiple threads to implement the reading and writing infrastructure. When reading or writing in a tight loop insert a Sleep call, passing 0, to allow a context switch to occur thereby giving the network variable threads time to execute.
... snip ...
NationalInstruments.NetworkVariable shares resources such as sockets and data buffers among connections that refer to the same network variable in the same program.
In my opinion, I'd expect better runtime performance by connecting/disconnecting as infrequently as possible. For example, when the network is reachable, connect; when it isn't, disconnect.
I am using Azure SDKs on IoT devices. One of the methods I rely on is
public Task<Message> ReceiveAsync();
which appears in this namespace
namespace Microsoft.Azure.Devices.Client
Under this class
public sealed class DeviceClient : IDisposable
I am calling this method continuously within a while loop as follows
while (true)
{
var receivedMessage = await _deviceClient.ReceiveAsync(TimeSpan.FromSeconds(3)).ConfigureAwait(false);
if (receivedMessage != null)
{
//Do staff
}
}
My question is: does this consume internet quotas even though the receivedMessage always shows null?
Digging through the source, you'll find three handlers:
HttpTransportHandler
MqttTransportHandler
AmqpTransportHandler
Which one is used, depends on your configuration. The HTTP one will issue a GET request per ReceiveAsync(), costing network traffic.
The MQTT handler operates on TCP or WebSockets, where keepalive traffic may be involved. But given this communication is bidirectional, most traffic that occurs involves actual messages being delivered. ReceiveAsync() simply gets the first message from the internal receive queue, if any, or waits for one to arrive, it doesn't poll.
The AMPQ handler also operates on a message queue, and I can't quite figure out whether a ReceiveAsync() will ultimately incur network traffic.
I am running a pub-sub set-up that works very well for a single publisher and multiple subscribers.
But I now wish to have several publishers publishing to the same "Channel", when I try this, the second time I try to Bind I get an address-already-used error.
Why can't I have a second publisher?
This is for a high-throughput application approx 250K messages/sec and a quick read of xPub-xSub suggests an intermediary will add overhead.
private void BackgroundProcess()
{
int msgSeqNum = 0;
using (var server = new PublisherSocket())
{
server.Options.SendHighWatermark = 1000;
server.Bind(Connection);
var address = Key;
string txt;
while (true)
{
if (O.TryTake(out txt, 60000))
{
msgSeqNum++;
server.SendMoreFrame(address).SendMoreFrame(msgSeqNum.ToString()).SendMoreFrame(DateTime.UtcNow.ToString("yyyyMMddTHHmmssffffff")).SendFrame("Whatever");
}
}
}
}
Netmq either works with a one on one socket set or a one to many socket set. You are getting close, you will need the xpub xsub to work as a proxy netmq actually provides one for this purpose.
https://netmq.readthedocs.io/en/latest/xpub-xsub/
As for why, this is a limitation of the underlying tcp layer, you can’t bind multiple tcp listeners to a single port afaik
I'm currently writing a small wrapper for NamedPipeServerStream/NamedPipeClientStream that is fully Event based as oppose to using AsyncCallbacks.
I expose sync and async methods for pretty much everything possible (connecting/waiting for connection, writing, etc) so if a consumer wanted to, for example, start a server instance and send a message when a client connects he could either go full sync route and do something like ...
var server = new NamedPipeServer("helloWorld");
server.StartAndWait();
server.Write("Welcome!");
or the async way like...
var server = new NamedPipeServer("helloWorld);
server.ClientConnected += x => x.WriteAsync("Welcome!");
server.Start(); //Start() returns immediately
However I'm struggling with finding a good way to do the same for reading messages. Currently when a message is read I fire a MessageAvailable event and pass the message in as one of the arguments.
I just can't come up with a proper way of implementing synchronous reads.
What I've considered is the following:
Having a GetNextMessage() sync method that gets the message. Internally, this could be handled in two different ways:
I could keep an IEnumerable<Message> with all of the not yet consumed messages. So as soon as the other side sends a message, I'd read it from the stream and store it in memory so they can be later consumed by GetNextMessage(). The advantage is that it frees up the stream pretty much as soon as the message is written, so it doesn't block the other side from sending other messages. The disadvantage is that I have absolutely no control over how many messages I'll be holding or the size of them. My IEnumerable<Message> might end up having 10GB worth of non-consumed messages, and there's nothing I can do about it since I can't force the consumer to retrieve messages.
I could take the view that I only ever store one message in an internal buffer, and only ever start reading again once that one was consumed via GetNextMessage(). If I do that though, the other side would be prevented from writing other messages until the previous one was consumed. To be more exact, the other side would be able to write until the stream is full. Which could be either multiple small complete messages or a single incomplete message. In the case of an incomplete single message I think this is a worse approach because in between part 1 of the message being sent and subsequent parts, the other end might end up disconnecting and the whole message will be lost.
To make things harder, in either of the approaches above there's always the chance that the consumer is using events for receiving messages (remember the event contains the message received) and therefore has no need for GetNextMessage(). I'd either need to stop sending the message in the event altogether, or find a way of not pushing the event to the internal buffer if the message is consumed via the event. And while I can easily tell whether there is an event handler or not, there's no way of knowing if the message is actually being handled there (i.e., consider a class implementing this one and listens to that event, yet does nothing with it). The only real approach I can see here is to remove the message from the event, force consumers to always call GetNextMessage(), but am open to other ideas.
There's also another problem with either of the approaches, which is the fact that I can't control the order in which the messages are sent if WriteAsync() is used (or Write() is used from different threads).
Can anyone think of a better way of tackling this problem?
I'd suggest the following approach. Create interface:
public interface ISubscription : IDisposable {
Message NextMessage(TimeSpan? timeout);
}
public class Message {
}
And then implement like that:
public class NamedPipeServer {
public void StartAndWait() {
}
public ISubscription StartAndSubscribe() {
// prevent race condition before Start and subscribing to MessageAvailable
var subscription = new Subscription(this);
StartAndWait();
return subscription;
}
public ISubscription Subscribe() {
// if user wants to subscribe and some point after start - why not
return new Subscription(this);
}
public event Action<Message> MessageAvailable;
private class Subscription : ISubscription {
// buffer
private readonly BlockingCollection<Message> _queue = new BlockingCollection<Message>(
new ConcurrentQueue<Message>());
private readonly NamedPipeServer _server;
public Subscription(NamedPipeServer server) {
// subscribe to event
_server = server;
_server.MessageAvailable += OnMessageAvailable;
}
public Message NextMessage(TimeSpan? timeout) {
// this is blocking call
if (timeout == null)
return _queue.Take();
else {
Message tmp;
if (_queue.TryTake(out tmp, timeout.Value))
return tmp;
return null;
}
}
private void OnMessageAvailable(Message msg) {
// add to buffer
_queue.Add(msg);
}
public void Dispose() {
// clean up
_server.MessageAvailable -= OnMessageAvailable;
_queue.CompleteAdding();
_queue.Dispose();
}
}
}
Client then either calls Subscribe or StartAndSubscribe.
var sub = server.StartAndSubscribe();
var message = sub.NextMessage();
var messageOrNull = sub.NextMessage(TimeSpan.FromSeconds(1));
sub.Dispose();
That way if no one subscribes - you buffer no messages. And if someone subscribes and then does not consume - it's their problem, not yours, because buffering happens in subscription they now own. You can also limit size of _queue blocking collection, then adding to it will block if limit is reached, blocking your MessageAvailable event, but I won't recommend doing that.
I'm using RabbitMQ in C# with the EasyNetQ library. I'm using a pub/sub pattern here. I still have a few issues that I hope anyone can help me with:
When there's an error while consuming a message, it's automatically moved to an error queue. How can I implement retries (so that it's placed back on the originating queue, and when it fails to process X times, it's moved to a dead letter queue)?
As far as I can see there's always 1 error queue that's used to dump messages from all other queues. How can I have 1 error queue per type, so that each queue has its own associated error queue?
How can I easily retry messages that are in an error queue? I tried Hosepipe, but it justs republishes the messages to the error queue instead of the originating queue. I don't really like this option either because I don't want to be fiddling around in a console. Preferably I'd just program against the error queue.
Anyone?
The problem you are running into with EasyNetQ/RabbitMQ is that it's much more "raw" when compared to other messaging services like SQS or Azure Service Bus/Queues, but I'll do my best to point you in the right direction.
Question 1.
This will be on you to do. The simplest way is that you can No-Ack a message in RabbitMQ/EasyNetQ, and it will be placed at the head of the queue for you to retry. This is not really advisable because it will be retried almost immediately (With no time delay), and will also block other messages from being processed (If you have a single subscriber with a prefetch count of 1).
I've seen other implementations of using a "MessageEnvelope". So a wrapper class that when a message fails, you increment a retry variable on the MessageEnvelope and redeliver the message back onto the queue. YOU would have to do this and write the wrapping code around your message handlers, it would not be a function of EasyNetQ.
Using the above, I've also seen people use envelopes, but allow the message to be dead lettered. Once it's on the dead letter queue, there is another application/worker reading items from the dead letter queue.
All of these approaches above have a small issue in that there isn't really any nice way to have a logarithmic/exponential/any sort of increasing delay in processing the message. You can "hold" the message in code for some time before returning it to the queue, but it's not a nice way around.
Out of all of these options, your own custom application reading the dead letter queue and deciding whether to reroute the message based on an envelope that contains the retry count is probably the best way.
Question 2.
You can specify a dead letter exchange per queue using the advanced API. (https://github.com/EasyNetQ/EasyNetQ/wiki/The-Advanced-API#declaring-queues). However this means you will have to use the advanced API pretty much everywhere as using the simple IBus implementation of subscribe/publish looks for queues that are named based on both the message type and subscriber name. Using a custom declare of queue means you are going to be handling the naming of your queues yourself, which means when you subscribe, you will need to know the name of what you want etc. No more auto subscribing for you!
Question 3
An Error Queue/Dead Letter Queue is just another queue. You can listen to this queue and do what you need to do with it. But there is not really any out of the box solution that sounds like it would fit your needs.
I've implemented exactly what you describe. Here are some tips based on my experience and related to each of your questions.
Q1 (how to retry X times):
For this, you can use IMessage.Body.BasicProperties.Headers. When you consume a message off an error queue, just add a header with a name that you choose. Look for this header on each message that comes into the error queue and increment it. This will give you a running retry count.
It's very important that you have a strategy for what to do when a message exceeds the retry limit of X. You don't want to lose that message. In my case, I write the message to disk at that point. It gives you lots of helpful debugging information to come back to later, because EasyNetQ automatically wraps your originating message with error info. It also has the original message so that you can, if you like, manually (or maybe automated, through some batch re-processing code) requeue the message later in some controlled way.
You can look at the code in the Hosepipe utility to see a good way of doing this. In fact, if you follow the pattern you see there then you can even use Hosepipe later to requeue the messages if you need to.
Q2 (how to create an error queue per originating queue):
You can use the EasyNetQ Advanced Bus to do this cleanly. Use IBus.Advanced.Container.Resolve<IConventions> to get at the conventions interface. Then you can set the conventions for the error queue naming with conventions.ErrorExchangeNamingConvention and conventions.ErrorQueueNamingConvention. In my case I set the convention to be based on the name of the originating queue so that I get a queue/queue_error pair of queues every time I create a queue.
Q3 (how to process messages in the error queues):
You can declare a consumer for the error queue the same way you do any other queue. Again, the AdvancedBus lets you do this cleanly by specifying that the type coming off of the queue is EasyNetQ.SystemMessage.Error. So, IAdvancedBus.Consume<EasyNetQ.SystemMessage.Error>() will get you there. Retrying simply means republishing to the original exchange (paying attention to the retry count you put in the header (see my answer to Q1, above), and information in the Error message that you consumed off the error queue can help you find the target for republishing.
I know this is an old post but - just in case it helps someone else - here is my self-answered question (I needed to ask it because existing help was not enough) that explains how I implemented retrying failed messages on their original queues. The following should answer your question #1 and #3. For #2, you may have to use the Advanced API, which I haven't used (and I think it defeats the purpose of EasyNetQ; one might as well use RabbitMQ client directly). Also consider implementing IConsumerErrorStrategy, though.
1) Since there can be multiple consumers of a message and all may not need to retry a msg, I have a Dictionary<consumerId, RetryInfo> in the body of the message, as EasyNetQ does not (out of the box) support complex types in message headers.
public interface IMessageType
{
int MsgTypeId { get; }
Dictionary<string, TryInfo> MsgTryInfo {get; set;}
}
2) I have implemented a class RetryEnabledErrorMessageSerializer : IErrorMessageSerializer that just updates the TryCount and other information every time it is called by the framework. I attach this custom serializer to the framework on a per-consumer basis via the IoC support provided by EasyNetQ.
public class RetryEnabledErrorMessageSerializer<T> : IErrorMessageSerializer where T : class, IMessageType
{
public string Serialize(byte[] messageBody)
{
string stringifiedMsgBody = Encoding.UTF8.GetString(messageBody);
var objectifiedMsgBody = JObject.Parse(stringifiedMsgBody);
// Add/update RetryInformation into objectifiedMsgBody here
// I have a dictionary that saves <key:consumerId, val: TryInfoObj>
return JsonConvert.SerializeObject(objectifiedMsgBody);
}
}
And in my EasyNetQ wrapper class:
public void SetupMessageBroker(string givenSubscriptionId, bool enableRetry = false)
{
if (enableRetry)
{
_defaultBus = RabbitHutch.CreateBus(currentConnString,
serviceRegister => serviceRegister.Register<IErrorMessageSerializer>(serviceProvider => new RetryEnabledErrorMessageSerializer<IMessageType>(givenSubscriptionId))
);
}
else // EasyNetQ's DefaultErrorMessageSerializer will wrap error messages
{
_defaultBus = RabbitHutch.CreateBus(currentConnString);
}
}
public bool SubscribeAsync<T>(Func<T, Task> eventHandler, string subscriptionId)
{
IMsgHandler<T> currMsgHandler = new MsgHandler<T>(eventHandler, subscriptionId);
// Using the msgHandler allows to add a mediator between EasyNetQ and the actual callback function
// The mediator can transmit the retried msg or choose to ignore it
return _defaultBus.SubscribeAsync<T>(subscriptionId, currMsgHandler.InvokeMsgCallbackFunc).Queue != null;
}
3) Once the message is added to the default error queue, you can have a simple console app/windows service that periodically republishes existing error messages on their original queues. Something like:
var client = new ManagementClient(AppConfig.BaseAddress, AppConfig.RabbitUsername, AppConfig.RabbitPassword);
var vhost = client.GetVhostAsync("/").Result;
var aliveRes = client.IsAliveAsync(vhost).Result;
var errQueue = client.GetQueueAsync(Constants.EasyNetQErrorQueueName, vhost).Result;
var crit = new GetMessagesCriteria(long.MaxValue, Ackmodes.ack_requeue_false);
var errMsgs = client.GetMessagesFromQueueAsync(errQueue, crit).Result;
foreach (var errMsg in errMsgs)
{
var innerMsg = JsonConvert.DeserializeObject<Error>(errMsg.Payload);
var pubInfo = new PublishInfo(innerMsg.RoutingKey, innerMsg.Message);
pubInfo.Properties.Add("type", innerMsg.BasicProperties.Type);
pubInfo.Properties.Add("correlation_id", innerMsg.BasicProperties.CorrelationId);
pubInfo.Properties.Add("delivery_mode", innerMsg.BasicProperties.DeliveryMode);
var pubRes = client.PublishAsync(client.GetExchangeAsync(innerMsg.Exchange, vhost).Result, pubInfo).Result;
}
4) I have a MessageHandler class that contains a callback func. Whenever a message is delivered to the consumer, it goes to the MessageHandler, which decides if the message try is valid and calls the actual callback if so. If try is not valid (maxRetriesExceeded/the consumer does not need to retry anyway), I ignore the message. You can choose to Dead Letter the message in this case.
public interface IMsgHandler<T> where T: class, IMessageType
{
Task InvokeMsgCallbackFunc(T msg);
Func<T, Task> MsgCallbackFunc { get; set; }
bool IsTryValid(T msg, string refSubscriptionId); // Calls callback only
// if Retry is valid
}
Here is the mediator function in MsgHandler that invokes the callback:
public async Task InvokeMsgCallbackFunc(T msg)
{
if (IsTryValid(msg, CurrSubscriptionId))
{
await this.MsgCallbackFunc(msg);
}
else
{
// Do whatever you want
}
}
Here, I have implemented a Nuget package (EasyDeadLetter) for this purpose, which can be easily implemented with the minimum changes in any project.
All you need to do is follow the four steps :
First of all, Decorate your class object with QeueuAttribute
[Queue(“Product.Report”, ExchangeName = “Product.Report”)]
public class ProductReport { }
The second step is to define your dead-letter queue with the same QueueAttribute and also inherit the dead-letter object from the Main object class.
[Queue(“Product.Report.DeadLetter”, ExchangeName =
“Product.Report.DeadLetter”)]
public class ProductReportDeadLetter : ProductReport { }
Now, it’s time to decorate your main queue object with the EasyDeadLetter attribute and set the type of dead-letter queue.
[EasyDeadLetter(DeadLetterType =
typeof(ProductReportDeadLetter))]
[Queue(“Product.Report”, ExchangeName = “Product.Report”)]
public class ProductReport { }
In the final step, you need to register EasyDeadLetterStrategy as the default error handler (IConsumerErrorStrategy).
services.AddSingleton<IBus>
(RabbitHutch.CreateBus(“connectionString”,
serviceRegister =>
{
serviceRegister.Register<IConsumerErrorStrategy,
EasyDeadLetterStrategy>();
}));
That’s all. from now on any failed message will be moved to the related dead-letter queue.
See more detail here :
GitHub Repository
NuGet Package