Develop Reference

Should I use Parallel.ForEach to loop through the DPS linkedHubs?

Here is the scenario:
DPS with enrollmentgroup with more than one linked IotHub
The enrollment group is associated with the custom function app.
Custom function app:
When the DPS request is made, a payload is sent to the DPS with the information that should determine what iothub(iothubhostname) the device should be registered to.
The app will receive the payload along with list of linkedIotHubhostname in the requestbody
App now needs to loop through the list of linkedIotHubHostname to determine which iothub the device belongs to based on the information provided.
For step 3, should I be using Parallel_foreach given the case that more than one device might be provisioning at the same time?

When multiple devices start provisioning at the same time, your Function will receive more requests. Depending on the function plan you chose, it will automatically create more instances to handle the load. The execution time of your Function will have some impact on when scaling is necessary, but unless you're talking execution time of seconds, Parallel.ForEach is not likely going to make a difference. It also depends on how many hubs and devices you're expecting to have.
You can deploy your function and check the execution time, optimise it later if necessary.

Dynamically set Endpoint definition in BizTalk Send Port using MQSeries adapter

Introduction
We exchange income data with an external party. Each year income tax regulations change and a new message schema has to be implemented. Altogether we now have 8 different schema versions each of which are deployed in a separate 'year income tax' application and this amount increases by 1 each year.
Because we pay our hosting company per installed application, we want to decrease the amount of applications installed.
All these applications are functionally equal, which means we validate incoming messages, and forward valid messages into a specific MQSeries queue. Each invalid message is routed to a response queue. Each application has it's own 'valid' and 'invalid' message queues.
The plan
One generic application that processes all 8(+) messages. New schemas must be deployable without application changes or downtime for previous, running 'income year tax' flows.
So far...
I can receive multiple messages on the same BizTalk receive port (MessageType XmlDocument) and am able to validate these messages dynamically in an orchestration by calling a custom receive pipeline (XML Disassembler + XML Validator). Exceptions as well as valid messages are processed as prescribed. There are no references between the Schemas and the generic application, so schemas can be deployed without need to stop running processes. So far, so good.
The orchestration has 1 receive shape, and 2 send shapes (valid, invalid).
SSO contains the values for routing the 'valid' and 'invalid' messages to their correct queue. Based on the incoming messagetype SSO is questioned for the correct 'valid' or 'invalid' queuedefinition.
The problem
I have previously dealt with dynamic FTP, FILE, WCF and SMTP ports, which all worked flawlessly after supplying the adapter with the correct Context Properties. Even MSMQ seems to have a fairly straightforward approach on dynamically setting transport properties.
However, I cannot seem to find MQSeries MQMT ContextProperties to set the queuedefinition dynamically.
Microsoft does not provide much information on this, and extensive searches on the internet hasn't provided me with anything useful (examples) either.
I tried matching IBM's docs with Microsoft's, but altogether I am now stuck.

I would suggest to use MQSC adapter for IBM MQ integration. It is part of Host Integration Server MSI. It only requires MQ client to be installed on the server Vs MQ Server for Windows installation required by MQSeries adapter.
Set the OutboundTransportLocation property in following format mqsc://{channelName}/tcp/{server{({port})/{queuemanager}/{queuename}
TransportType = MQSC
Context Properties - Schema can be found within assembly MQSeriesEx.MQSPropertySchemaEx with namespace (http://schemas.microsoft.com/BizTalk/2003/mqs-properties).
There are only few context properties you would need to set if at all required.
Channel_HeartBeat
Channel_MaxMessageLength
Channel_UserId
Channel_Password
ConnectionTimeout
If additional properties are required than use MQSeries.MQSPropertySchema context properties.

Thanks Vikas for your suggestion.
I followed your directions and found it works!
However, I found it a little more complicated than needed as it required me configuring channel names for each flow.
The solution that best suited me was the one I had in mind all along, and it was right before me. My attempts failed because I made a fatal mistake by setting the outgoing message's properties where I should have set the dynamic send port's properties.
SendPort(Microsoft.XLANGs.BaseTypes.Address)="MQS://SERVER/QMANAGER/QUEUENAME";

How to get programmatically limit message storage which it is in Message Queueing Properties?

How to get programmatically limit message storage which it is in Message Queueing Properties?
It defines size of all queues - global quota.
I am creating a simple monitor program which monitoring sizes of queues. I use MSMQ COM in C#, and i can't find that property. I can only find out how many bytes are in all queues and how many bytes is in particular queue. Default size (quota ) of queue is set on -1 which means infinity.

Modbus Custom Message & Sub functions (NModbus, TCP, Subfunctions)

I am using NModbus in a C# project, to read & write Modbus data from/to a number of I/O devices. I am using the Modbus TCP/IP protocol (ModbusIpMaster etc..) in the program.
I have successfully communicated with the devices (through a Modbus gateway) and can use the default methods (e.g. common Modbus functions such as ReadHoldingRegisters, WriteCoils ), to access the data from the devices, and can write back to them. At the moment, all I can do are the default NModbus methods which expose commonly used Modbus codes (1, 2, 3, 4 etc..).
I have two difficulties:
1) The I/O devices' settings and extra information can be accessed under the Modbus code of 70 (0x46), and there are sub functions which I will need to use in order to read and/or write the settings.
For example, Func 07 (0x46), Sub func 6 (0x06), can be used to set the communications settings of a module. In this example there are 7 bytes of information to be sent which carry the settings (e.g. baud rate etc...)
NModbus does not have a specific method for this 'custom' function code (70). So, from what I understand, one needs to use the CustomMessage feature of NModbus. I have tried executing CustomMessage, and when using the common Modbus function codes (e.g. 1 or 2) I can achieve the same result as if I were using the default methods of NModbus (i.e. the CustomMessage is working so far).
When I try function codes other than the general ones (e.g. 1, 2, 3...) I do get all sorts of exceptions. Furthermore, I do not know how I should send the sub-function with the message!
When I added the subfunction just after the function code (e.g. 70 followed by 06), and the data is send through I get, exceptions. I really need help from experts in this field, please.
Here is a more obvious exception:
" Exception of type 'Modbus.SlaveException' was thrown. Function Code: 198 Exception Code: 3 - A value contained in the query data
field is not an allowable value for server (or slave). This indicates
a fault in the structure of the remainder of a complex request, such
as that the implied length is incorrect. It specifically does NOT mean
that a data item submitted for storage in a register has a value
outside the expectation of the application program, since the MODBUS
protocol is unaware of the significance of any particular value of any
particular register."
2) The second problem is sending ASCII RS-232 messages through Modbus, in order to control an RS-232 device which is connected to COM1 of a module which has the ability to convert Modbus messages. In other words, I am planning to communicate with the Modbus gateway, to send ASCII data to its COM1, which in turn translates data to RS-232 and then communicates with the RS-232 device. The translation is meant to occur internal to the gateway, so all I need to know is how on Earth I can send these messages through, which vary in length. I have no idea how that is possible, and where to start from.

Part 2 - NModbus allows you to create an ASCII master over a TCP or UDP client.
using (TcpClient client = new TcpClient("127.0.0.1", 502))
{
ModbusSerialMaster master = ModbusSerialMaster.CreateAscii(client);
// Use the master here
}
With regard to the first part of the question, NModbus is great for the basics, but it lacks a layer of abstraction that would allow it to be extended arbitrarily. My own experience of it ended when I found that it would require significant modification to add and make the GetServerId function to work, even though this is part of the specification for Modbus RTU. My solution was to re-implement it from the ground up, (which took me less time than getting the NModbus source to compile and run in the first place, due to it's dependency on third part libraries for logging, etc & missing extensions for IEnumerable classes).

Suggestions for developing a TCP/IP based message client

I've got a server side protocol that controls a telephony system, I've already implemented a client library that communicates with it which is in production now, however there are some problems with the system I have at the moment, so I am considering re-writing it.
My client library is currently written in Java but I am thinking of re-writing it in both C# and Java to allow for different clients to have access to the same back end.
The messages start with a keyword have a number of bytes of meta data and then some data. The messages are always terminated by an end of message character.
Communication is duplex between the client and the server usually taking the form of a request from the Client which provokes several responses from the server, but can be notifications.
The messages are marked as being on of:
C: Command
P: Pending (server is still handling the request)
D: Data data as a response to
R: Response
B: Busy (Server is too busy to handle response at the moment)
N: Notification
My current architecture has each message being parsed and a thread spawned to handle it, however I'm finding that some of the Notifications are processed out of order which is causing me some trouble as they have to be handled in the same order they arrive.
The duplex messages tend to take the following message format:
Client -> Server: Command
Server -> Client: Pending (Optional)
Server -> Client: Data (optional)
Server -> Client: Response (2nd entry in message data denotes whether this is an error or not)
I've been using the protocol for over a year and I've never seen the a busy message but that doesn't mean they don't happen.
The server can also send notifications to the client, and there are a few Response messages that are auto triggered by events on the server so they are sent without a corresponding Command being issued.
Some Notification Messages will arrive as part of sequence of messages, which are related for example:
NotificationName M00001
NotificationName M00001
NotificationName M00000
The string M0000X means that either there is more data to come or that this is the end of the messages.
At present the tcp client is fairly dumb it just spawns a thread that notifies an event on a subscriber that the message has been received, the event is specific to the message keyword and the type of message (So data,Responses and Notifications are handled separately) this works fairly effectively for Data and response messages, but falls over with the notification messages as they seem to arrive in rapid sequence and a race condition sometimes seems to cause the Message end to be processed before the ones that have the data are processed, leading to lost message data.
Given this really badly written description of how the system works how would you go about writing the client side transport code?
The meta data does not have a message number, and I have not control over the underlying protocol as it's provided by a vendor.

The requirement that messages must be processed in the order in which they're received almost forces a producer/consumer design, where the listener gets requests from the client, parses them, and then places the parsed request into a queue. A separate thread (the consumer) takes each message from the queue in order, processes it, and sends a response to the client.
Alternately, the consumer could put the result into a queue so that another thread (perhaps the listener thread?) can send the result to the client. In that case you'd have two producer/consumer relationships:
Listener -> event queue -> processing thread -> output queue -> output thread
In .NET, this kind of thing is pretty easy to implement using BlockingCollection to handle the queues. I don't know if there is something similar in Java.
The possibility of a multi-message request complicates things a little bit, as it seems like the listener will have to buffer messages until the last part of the request comes in before placing the entire thing into the queue.
To me, the beauty of the producer/consumer design is that it forces a hard separation between different parts of the program, making each much easier to debug and minimizing the possibility of shared state causing problems. The only slightly complicated part here is that you'll have to include the connection (socket or whatever) as part of the message that gets shared in the queues so that the output thread knows where to send the response.
It's not clear to me if you have to process all messages in the order they're received or if you just need to process messages for any particular client in the proper order. For example, if you have:
Client 1 message A
Client 1 message B
Client 2 message A
Is it okay to process the first message from Client 2 before you process the second message from Client 1? If so, then you can increase throughput by using what is logically multiple queues--one per client. Your "consumer" then becomes multiple threads. You just have to make sure that only one message per client is being processed at any time.

I would have one thread per client which does the parsing and processing. That way the processing would be in the order it is sent/arrives.
As you have stated, the tasks cannot be perform in parallel safely. performing the parsing and processing in different threads is likely to add as much overhead as you might save.
If your processing is relatively simple and doesn't depend on external systems, a single thread should be able to handle 1K to 20K messages per second.
Is there any other issues you would want to fix?

I can recommend only for Java-based solution.
I would use some already mature transport framework. By "some" I mean the only one I have worked with until now -- Apache MINA. However, it works and it's very flexible.
Regarding processing messages out-of-order -- for messages which must be produced in the order they were received you could build queues and put such messages into queues.
To limit number of queues, you could instantiate, say, 4 queues, and route incoming message to particular queue depending on the last 2 bits (indeces 0-3) of the hash of the ordering part of the message (for example, on the client_id contained in the message).
If you have more concrete questions, I can update my answer appropriately.