I have a request/response protocol that runs over TCP that I'd like to provide an async/await API for. The protocol is STOMP, which is a fairly simple text-based protocol that runs over TCP or SSL. In STOMP, the client sends one of six or so command frames and specifies a receipt ID in the header of the command. The server will respond with either a RECEIPT or ERROR frame, with a receipt-id field, so the client can match the response with the original request. The server can also send a MESSAGE frame at any time (STOMP is fundamentally a messaging protocol) which will not contain a receipt-id.
To allow multiple outstanding requests and handle any MESSAGE frames, the plan is to always have a Socket.BeginReceive() outstanding. So what I was thinking is that the easiest implementation would be to create a waitable event (like a mutex), store that event in a table, send the command request with the receipt set to the index into the table, and block on the event. When socket.BeginReceive() fires the function can get the receipt-id from the message, look up the event in the table, and signal it (and store some state, like success or error). This will wake up the calling function, which can look at the result and return success or failure to the calling application.
Does this sound fundamentally correct? I've used async/await APIs before but have never written my own. If it's OK what kind of waitable event should I use? A simple Monitor.Wait() will block but not in the way I want, correct? If I wrap the whole thing in Task.Run() will that behave properly with Monitor.Wait()? Or is there a new synchronization construct that I should be using instead? I'm basically implementing HttpClient.GetAsync(), does anyone know how that works under the covers?
HttpClient is much simpler, because HTTP only has one response for each request. There's no such thing as an unsolicited server message in HTTP.
To properly set up a "stream" of events like this, it's best to use TPL Dataflow or Rx. Otherwise, you'd have to create an unbounded receive buffer and have repeated async ReceiveMessage calls.
So I'd recommend using a TPL Dataflow pipeline to create a source block of "messages", and then matching some up with requests (using TaskCompletionSource to notify the sender it's complete) and exposing the rest (MESSAGE frames) as a source block.
Internally, your processing pipeline would look like this:
Repeated BeginReceive ->
TransformBlock for message framing ->
ActionBlock to match response messages to requests.
BufferBlock for MESSAGE frames.
Related
I'm using NServiceBus with RabbitMQ in my project. I have two services that don't know about each other and don't share anything. service1 publishes request messages to endpoint1 (queue1) and service2 listens to endpoint1 and publishes responses to endpoint2 (queue2). There are two questions:
How can service1 handle responses from service2 if service1 doesn't know the response message type but only expects some particular fields in the response message?
I want to create an async API method that sends a request to endpoint1 and waits for the response in endpoint2. Is it somehow possible at all? Also how can I ensure that the reply corresponds with the request?
I expect something like:
public async Task<object> SendRequest(string str) {
var request = new MyRequest(str);
await endPoint1.Publish(request);
var reply = await endPoint2.WaitingReply();
return reply;
}
I will appreciate any help.
Whenever two things communicate, there is always a contract. When functions call each other the contract is the parameters that are required to call that function. With messaging the message is the contract. The coupling is towards the message, not the sender or receiver.
I'm not really sure what you're trying to achieve? You mention an API which is async and endpoint1 and endpoint2.
First of all, there's asynchronous execution and asynchronous communication. The async part in your example code is asynchronous execution of two methods that have the word await in front of them. When we talk about sending messages, that's asynchronous communication. A message is put on the queue and then the code moves on and never looks back at the message. Even when you use the request/reply pattern, no code is actually waiting for a message.
You can wait for a message by blocking the thread, but I highly recommend you avoid that and not use the NServiceBus callback feature. If you think you have to, think again. If you still think so, read the red remarks on that page. If they can't convince you, contact Particular Software to have them explain another time why not. ;-)
It could be that you need a reply message for whatever reason. If you build some website using SignalR (for example) and you want to inform the user on the website when a message returned and some work was completed, you can wait for a reply message. The result is that the website itself becomes an endpoint.
So if the website is EndpointA and it sends a message to EndpointB, it is possible to reply to that message. EndpointA would then also need a message handler for that message. If EndpointB first needs to send a message to EndpointC, which in turn responds to EndpointB and only then it replies back to EndpointA, NServiceBus can't easily help. Not because it's impossible, but because you probably need another solution. EndpointA should probably not be waiting for that many endpoints to reply, so many things could go "wrong" and take too much time.
If you're interested to see how replies work in combination with SignalR and what not, you can check a demo I built for a presentation that has that.
I have an array of websites that (asynchronously) send event analytics into an ASP.NET website, which then should send the events into an Azure EventHubs instance.
The challenge I'm facing is that with requests exceeding 50,000 per second I've noticed that my response times to serve these requests are into the multi-second range, effecting total load times for the initial sending website. I have scaled up all parts however I recognize that sending an event per request is not very efficient due to the overhead of opening an AMQP connection to Event Hubs and sending off the payload.
As a solution I've been trying to batch the Event Data that gets sent to my EventHubs instance however I've been running into some problems with synchronizing.
With each request, I add the Event Data into a static EventDataBatch created via EventHubClient.CreateBatch() with eventHubData.TryAdd() then I check to see that the quantity of events is within a predefined threshold and if so, I send the events asynchronously via EventHubClient.SendAsync(). The challenge this has created is that since this is a ASP .NET application, there could be many threads attempting to serve requests at any given instance - any of which could be trying to to eventHubData.TryAdd() or EventHubClient.SendAsync() at the same point in time.As a poor attempt to resolve this I have attempted to call lock(batch) prior to eventHubData.TryAdd() however this does not resolve the issue since I cannot also lock the asynchronous method EventHubClient.SendAsync().
What is the best way to implement this solution so that each request does not require it's own request to Event hubs and can take advantage of batching while also preserving the integrity of the batch itself and not running into any deadlock issues?
Have a look at the source code for the application insights SDK to see how they have solved this problem - you can reuse the key parts of this to achieve the same thing with event hubs AMQP.
The pattern is ,
1) Buffer data. Define a buffer that you will share among threads with a maximum size. Multiple threads write data into the buffer
https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/src/Microsoft.ApplicationInsights/Channel/TelemetryBuffer.cs
2) Prepare a transmission. You can transmit the items in the buffer either when the buffer is full, when some interval elapses, or whichever happens first. Take all the items from the buffer to send
https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/src/Microsoft.ApplicationInsights/Channel/InMemoryTransmitter.cs
3) Do the transmission. Send all items as multiple data points in a single Event Hub message,
https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/src/Microsoft.ApplicationInsights/Channel/Transmission.cs
They are the 3 classes that combine to achieve this using HTTP to post to the Application Insights collection endpoint - you can see how the sample pattern can be applied to collect, amalgamate and transmit to Event Hubs.
You'll need to control the maximum message size, which is 256KB per Event Hub message, which you could do by setting the telemetry buffer size - that's up to your client logic to manage that.
I'm working on an Asynchronous HTTP handler and trying to figure out if the HttpResponse.Write function blocks until it receives an ACK from the client.
The MSDN documentation doesn't specifically say; however, I do know that the MSDN documentation for the ISAPI WriteClient() function (a similar mechanism) mentions that the synchronous version does block while attempting to send data to the client.
I thought of three possible ways to determine the answer:
Have someone tell me its non-blocking
Write a low level TCP test client and set break point on the acknowledgement ( is this possible?)
Use reflection to inspect the inner workings of the HTTPResponse.Write method ( is this possible?)
Its not blocking, but can use a buffer and send them all together.
Try to set HttpResponse.Buffer=false; to direct write to your client.
You can also use the HttpResponse.Flush(); to force to send what you have to your client.
About HttpResponse.Buffer Property on MSDN
And maybe this intresting you: Web app blocked while processing another web app on sharing same session
HttpResponse operates in two distinct modes, buffered and unbuffered. In buffered mode, the various Write functions put their data into a memory region and the function returns as soon as the data is copied over. If you set Buffer to false, Write blocks until all of the data is sent to the client.
I've got a server side protocol that controls a telephony system, I've already implemented a client library that communicates with it which is in production now, however there are some problems with the system I have at the moment, so I am considering re-writing it.
My client library is currently written in Java but I am thinking of re-writing it in both C# and Java to allow for different clients to have access to the same back end.
The messages start with a keyword have a number of bytes of meta data and then some data. The messages are always terminated by an end of message character.
Communication is duplex between the client and the server usually taking the form of a request from the Client which provokes several responses from the server, but can be notifications.
The messages are marked as being on of:
C: Command
P: Pending (server is still handling the request)
D: Data data as a response to
R: Response
B: Busy (Server is too busy to handle response at the moment)
N: Notification
My current architecture has each message being parsed and a thread spawned to handle it, however I'm finding that some of the Notifications are processed out of order which is causing me some trouble as they have to be handled in the same order they arrive.
The duplex messages tend to take the following message format:
Client -> Server: Command
Server -> Client: Pending (Optional)
Server -> Client: Data (optional)
Server -> Client: Response (2nd entry in message data denotes whether this is an error or not)
I've been using the protocol for over a year and I've never seen the a busy message but that doesn't mean they don't happen.
The server can also send notifications to the client, and there are a few Response messages that are auto triggered by events on the server so they are sent without a corresponding Command being issued.
Some Notification Messages will arrive as part of sequence of messages, which are related for example:
NotificationName M00001
NotificationName M00001
NotificationName M00000
The string M0000X means that either there is more data to come or that this is the end of the messages.
At present the tcp client is fairly dumb it just spawns a thread that notifies an event on a subscriber that the message has been received, the event is specific to the message keyword and the type of message (So data,Responses and Notifications are handled separately) this works fairly effectively for Data and response messages, but falls over with the notification messages as they seem to arrive in rapid sequence and a race condition sometimes seems to cause the Message end to be processed before the ones that have the data are processed, leading to lost message data.
Given this really badly written description of how the system works how would you go about writing the client side transport code?
The meta data does not have a message number, and I have not control over the underlying protocol as it's provided by a vendor.
The requirement that messages must be processed in the order in which they're received almost forces a producer/consumer design, where the listener gets requests from the client, parses them, and then places the parsed request into a queue. A separate thread (the consumer) takes each message from the queue in order, processes it, and sends a response to the client.
Alternately, the consumer could put the result into a queue so that another thread (perhaps the listener thread?) can send the result to the client. In that case you'd have two producer/consumer relationships:
Listener -> event queue -> processing thread -> output queue -> output thread
In .NET, this kind of thing is pretty easy to implement using BlockingCollection to handle the queues. I don't know if there is something similar in Java.
The possibility of a multi-message request complicates things a little bit, as it seems like the listener will have to buffer messages until the last part of the request comes in before placing the entire thing into the queue.
To me, the beauty of the producer/consumer design is that it forces a hard separation between different parts of the program, making each much easier to debug and minimizing the possibility of shared state causing problems. The only slightly complicated part here is that you'll have to include the connection (socket or whatever) as part of the message that gets shared in the queues so that the output thread knows where to send the response.
It's not clear to me if you have to process all messages in the order they're received or if you just need to process messages for any particular client in the proper order. For example, if you have:
Client 1 message A
Client 1 message B
Client 2 message A
Is it okay to process the first message from Client 2 before you process the second message from Client 1? If so, then you can increase throughput by using what is logically multiple queues--one per client. Your "consumer" then becomes multiple threads. You just have to make sure that only one message per client is being processed at any time.
I would have one thread per client which does the parsing and processing. That way the processing would be in the order it is sent/arrives.
As you have stated, the tasks cannot be perform in parallel safely. performing the parsing and processing in different threads is likely to add as much overhead as you might save.
If your processing is relatively simple and doesn't depend on external systems, a single thread should be able to handle 1K to 20K messages per second.
Is there any other issues you would want to fix?
I can recommend only for Java-based solution.
I would use some already mature transport framework. By "some" I mean the only one I have worked with until now -- Apache MINA. However, it works and it's very flexible.
Regarding processing messages out-of-order -- for messages which must be produced in the order they were received you could build queues and put such messages into queues.
To limit number of queues, you could instantiate, say, 4 queues, and route incoming message to particular queue depending on the last 2 bits (indeces 0-3) of the hash of the ordering part of the message (for example, on the client_id contained in the message).
If you have more concrete questions, I can update my answer appropriately.
I'm using ActiveMQ in a .Net program and I'm flooded with message-events.
In short when I get a queue-event 'onMessage(IMessage receivedMsg)' I put the message into an internal queue out of which X threads do their thing.
At first I had: 'AcknowledgementMode.AutoAcknowledge' when creating the session so I'm guessing that all the messages in the queue got sucked down and put into the memory queue (which is risky since with a crash, everything is lost).
So then I used: 'AcknowledgementMode.ClientAcknowledge' when creating the session, and when a worker was ready with the message it calls the 'commit()' method on the message. However, still all the messages get sucked down from the queue.
How can I configure it that ONLY an X amount of messages are being processed or are in an internal queue, and that not everything is being 'downloaded' right away?
Are you on .NET 4.0? You could use a BlockingCollection . Set it to the maximum amount it may contain. As soon as a thread tries to put in an excess element, the Add operation will block until the collection falls below the threshold again.
Maybe that would do it for throttling?
There is also an API for throttling in the Rx framework, but I do not know how it is implemented. If you implement your Queue source as Observable, this API would become available for you, but I don't know if this hits your needs.
You can set the client prefetch to control how many messages the client will be sent. When the Session is in Auto Ack, the client will only ack a message once its been delivered to your app via the onMessage callback or through a synchronous receive. By default the client will prefetch 1000 messages from the broker, if the client goes down these messages would be redelivered to another client it this was a Queue, otherwise for a topic they are just discarded as a topic is a broadcast based channel. If you set the prefetch to one then you client would only be sent one message from the sever, then each time your onMessage callback completes a new message would be dispatched as the client would ack that message, that is if the session is in Auto Ack mode.
Refer to the NMS configuration page for all the options:
http://activemq.apache.org/nms/configuring.html
Regards
Tim.
FuseSource.com