Develop Reference

Receive multipart response and treat each part as soon as received

Current situation: an existing SQL Server stored procedure I have no control upon returns 10 large strings in separate resultsets in about 30 seconds (~3 seconds per dataset). The existing ASP.NET Web API controller method that collects these strings only returns a response once all strings are obtained from the stored procedure. When the client receives the response, it takes another 30 seconds to process the strings and display the results, for a total of 1 minute from request initiation to operation completion.
Contemplated improvement: somehow transmit the strings to the client as soon as each is obtained from the SqlDataReader, so the client can work on interpreting each string while receiving the subsequent ones. The total time from request initiation to completion would thus roughly be halved.
I have considered the WebClient events at my disposal, such as DownloadStringCompleted and DownloadProgressChanged, but feel none is viable and generally think I am on the wrong track, hence this question. I have all kinds of ideas, such as saving strings to temporary files on the server and sending each file name to the client through a parallel SignalR channel for the client to request in parallel, etc., but feel I would both lose my time and your opportunity to enlighten me.

I would not resort to inverting the standard client / server relationship using a "server push" approach. All you need is some kind of intermediary dataset. It could be a singleton object (or multiple objects, one per client) on your server, or another table in an actual database (perhaps NoSql).
The point is that the client will not directly access the slow data flow you're dealing with. Instead the client will only access the intermediary dataset. On the first request, you will start off the process of migrating data from the slow dataset to the intermediary database and the client will have to wait until the first batch is ready.
The client will then make additional requests as he processes each result on his end. If more intermediary results are already available he will get them immediately, otherwise he will have to wait like he did on the first request.
But the server is continuously waiting on the slow data set and adding more data to the intermediate data set. You will have to have a way of marking the intermediate data as having already been sent to the client or not. You will probably want to spawn a separate thread for the code that moves data from the slow data source to the intermediate one.

Batching to Event Hubs from an ASP .NET Application

I have an array of websites that (asynchronously) send event analytics into an ASP.NET website, which then should send the events into an Azure EventHubs instance.
The challenge I'm facing is that with requests exceeding 50,000 per second I've noticed that my response times to serve these requests are into the multi-second range, effecting total load times for the initial sending website. I have scaled up all parts however I recognize that sending an event per request is not very efficient due to the overhead of opening an AMQP connection to Event Hubs and sending off the payload.
As a solution I've been trying to batch the Event Data that gets sent to my EventHubs instance however I've been running into some problems with synchronizing.
With each request, I add the Event Data into a static EventDataBatch created via EventHubClient.CreateBatch() with eventHubData.TryAdd() then I check to see that the quantity of events is within a predefined threshold and if so, I send the events asynchronously via EventHubClient.SendAsync(). The challenge this has created is that since this is a ASP .NET application, there could be many threads attempting to serve requests at any given instance - any of which could be trying to to eventHubData.TryAdd() or EventHubClient.SendAsync() at the same point in time.As a poor attempt to resolve this I have attempted to call lock(batch) prior to eventHubData.TryAdd() however this does not resolve the issue since I cannot also lock the asynchronous method EventHubClient.SendAsync().
What is the best way to implement this solution so that each request does not require it's own request to Event hubs and can take advantage of batching while also preserving the integrity of the batch itself and not running into any deadlock issues?

Have a look at the source code for the application insights SDK to see how they have solved this problem - you can reuse the key parts of this to achieve the same thing with event hubs AMQP.
The pattern is ,
1) Buffer data. Define a buffer that you will share among threads with a maximum size. Multiple threads write data into the buffer
https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/src/Microsoft.ApplicationInsights/Channel/TelemetryBuffer.cs
2) Prepare a transmission. You can transmit the items in the buffer either when the buffer is full, when some interval elapses, or whichever happens first. Take all the items from the buffer to send
https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/src/Microsoft.ApplicationInsights/Channel/InMemoryTransmitter.cs
3) Do the transmission. Send all items as multiple data points in a single Event Hub message,
https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/src/Microsoft.ApplicationInsights/Channel/Transmission.cs
They are the 3 classes that combine to achieve this using HTTP to post to the Application Insights collection endpoint - you can see how the sample pattern can be applied to collect, amalgamate and transmit to Event Hubs.
You'll need to control the maximum message size, which is 256KB per Event Hub message, which you could do by setting the telemetry buffer size - that's up to your client logic to manage that.

What is the difference between RequestInterceptor and MessageInspector?

I have two queries here :-
1) What is the basic difference between Microsoft.ServiceModel.Web.RequestInterceptor and System.ServiceModel.Dispatcher.DispatchRuntime.MessageInspectors (IdispatchMessageInterceptor)
Both appears to be Request/Message interceptors which can be used to implement custom validation/interceptors in the request pipeline.
When to use one on another?
2) Also how to plugin RequestInterceptor in RouteTable.Routes.Add(new ServiceRoute())
I have class like this -
public class AuthenticationInterceptor : RequestInterceptor
{
//Authentication logic goes here......
}
and a route definition like this : -
RouteTable.Routes.Add(new ServiceRoute(routePrefix, new MyServiceHostFactory(container, (sh) => {
foreach (System.ServiceModel.Dispatcher.ChannelDispatcher cd in sh.ChannelDispatchers)
{
foreach (System.ServiceModel.Dispatcher.EndpointDispatcher ed in cd.Endpoints)
{
ed.DispatchRuntime.MessageInspectors.Add(new AuthenticationInterceptor());
}
}
return sh; })));
Here is the definition of MyServiceHostFactory: -
public MyServiceHostFactory(IocContainer container, Func<ServiceHost, ServiceHost> createservicehost = null);
Now it is throwing following error : -
The best overloaded method match for 'System.Collections.Generic.SynchronizedCollection<System.ServiceModel.Dispatcher.IDispatchMessageInspector>.Add(System.ServiceModel.Dispatcher.IDispatchMessageInspector)' has some invalid arguments
at this line : -
ed.DispatchRuntime.MessageInspectors.Add(new AuthenticationInterceptor());
I know why, it is just because I am trying to hookup RequestInterceptor in MessageInspector. Both comes in different interface hierarchy.
So what should I do here?
Edit:
Also note, I cannot change AuthenticationInterceptor logic as the code in not under my control.

Here are the answers to your questions(you need to read through point number 2 to understand Interceptors and Inspectors a little):
1. Solution to the Error (you need to add your code logic to it )
Implement IDispatchMessageInspector in the following code. Please note that the name of the following class should change to inspector, but as you have mentioned you cannot change it so you should implement the interface here. Otherwise, it is recommended to create another class with Matching Inspector suffix and implementation.
public class AuthenticationInterceptor : RequestInterceptor, IDispatchMessageInspector
{
//Authentication logic goes here......
object IDispatchMessageInspector.AfterReceiveRequest(ref System.ServiceModel.Channels.Message request, System.ServiceModel.IClientChannel channel, System.ServiceModel.InstanceContext instanceContext)
{
//Your code here.
}
void IDispatchMessageInspector.BeforeSendReply(ref System.ServiceModel.Channels.Message reply, object correlationState)
{
//Your code here.
}
}
2. Difference between RequestInterceptor and MessageInspectors
In any client server communication there can be two important phases of communication. First, when a client establishes connection with the server and second when they both communicate.
When establishing the connection it is not necessary that the client which is trying to establish the connection is a valid client. It could be an unauthorized request as well or it could be possible that the request is valid but not intended to the destined server and requires authorization or connection rerouting.
A good example of rerouting is when :
you want regional client/servers to avoid cross region communications but one of the client(which is valid) but tries to connect to a different region server.
You want the servers selectively decide if you wish to allow cross region client-server communications for a few exceptional users.
There could be even more complex rerouting scenarios which is out of scope of this answer.
So in WCF, the Rest starter kit provides you additional ability to intercept the request during connection establishment phase. The interceptor (in your case the AuthenticationInterceptor) should authenticate such requests and if the request is invalid, it can log the necessary entries and just reject to process any communication further from this rejected client/session.
We have many benefits of having a RequestInterceptor:
It helps us to validate the incoming request at a very early phase.
It can help us to build custom authenticators or rerouting components.
It blocks any further message processing during request phase itself which is very important to keep the unnecessary load away from the WCF Service/Server.
Message Inspectors:
The MessageInspectors can be treated as part of second phase of client-server communication when a Request is validated and connection is well established and thus it's the time when client-server has to started communicating by passing messages to each other. Now, in your application environment it could be possible that the messages pass using a binary, xml or json serialized format. There could be applicable encryptions.
An example is, it is possible that a message arrives from client A and given to Server B now server Queues it up to another Server C which can wait for some more information from Another Server D. Once Server D provides the information, the Server C which has the message in queue further joins the raw message received from Server B and Server D, gives it to another service to deserialize and convert it to something meaningful which can be returned to server B and B returns it back to Client A.
Quite complex, right? But a multi server authentication like payments by credit card using mobile PIN somewhat works in similar manner, though could be not exactly the same, but even more complex.
In WCF, Interceptors and Inspectors can work together and their duties are different. An interceptor validates the end user/connection/rerouting and an Inspector validates/processes the message.
A few points:
You can build your own message inspectors by implementing IClientMessageInspector for the client side and IDispatchMessageInspector on the server side.
You can implement both of the interfaces in a single class if you are owner of both client and server components.
Here, in your case it seems you need to implement IDispatchMessageInspector.
A class implementing IDispatchMessageInspector does not intercepts as I mentioned before but is meant to 'Inspect' an incoming message and any outgoing message and this Inspector can be hooked using configurations when a message arrives from the client.
Please note that by this time at Inspector levels, any message which arrives is already processed at various channel stack levels and assigned to which WCF Service will process this request. If you are using any encryption in between, then the message has already been decrypted. But, the message is not deserialized yet.
A use of having your custom Inspector is it could be possible that your system implements custom serialization format like (SWIFT/FIX protocol in banking) or another level of zip/unzip encoding etc.
This custom inspector can deserialized the data and give it to your component COMP which is actually meant to work on the deserialized data.
An IDispatchMessageInspector interface has got two methods which you need to implement:
a) AfterReceiveRequest and
b) BeforeSendReply(ref Message, Object).
AfterReceiveRequest is the method which can dezerialize the data and gives it to COMP, and BeforeSendReply is the method which again serializes the data and perform any operation on the message.
You can use behaviors to attach the MessageInspectors for every message your web service receives.
Both, custom Interceptors and Inspectors are largely meant to be used in Enterprise Platform or a Highly customizable platforms.
Hope this answer helps you. You can read more on the following links (possible you have gone through the first one):
http://msdn.microsoft.com/en-us/library/ee391967.aspx
http://msdn.microsoft.com/en-us/library/aa717047(v=vs.110).aspx
Regards
Kajal

TCPListener / TCPClient Server-Client Writing / Reading data

Here I am troubleshooting a theoretical problem about HOW servers and clients are working on machines. I know all NET Processes, but I am missing something referring to code. I was unable to find something related about this.
I code in Visual C# 2008, i use regular TCPClient / TCPListener with 2 different projects:
Project1 (Client)
Project2 (Server)
My issues are maybe so simple:
1-> About how server receives data, event handlers are possible?
In my first server codes i used to make this loop:
while (true)
{
if (NetworkStream.DataAvailable)
{
//stuff
}
Thread.Sleep(200);
}
I encounter this as a crap way to control the incoming data from a server. BUT server is always ready to receive data.
My question: There is anything like...? ->
AcceptTcpClient();
I want a handler that waits until something happen, in this case a specific socket data receiving.
2-> General networking I/O methods.
The problem is (beside I'm a noob) is how to handle multiple data writing.
If I use to send a lot of data in a byte array, the sending can break if I send more data. All data got joined and errors occurs when receiving. I want to handle multiple writes to send and receive.
Is this possible?

About how server receives data, event handlers are possible?
If you want to write call-back oriented server code, you may find MSDN's Asynchronous Server Socket Example exactly what you're looking for.
... the sending can break if I send more data. All data got joined and errors occurs when receiving.
That is the nature of TCP. The standardized Internet protocols fall into a few categories:
block oriented stream oriented
reliable SCTP TCP
unreliable UDP ---
If you really want to send blocks of data, you can use SCTP, but be aware that many firewalls DROP SCTP packets because they aren't "usual". I don't know if you can reliably route SCTP packets across the open Internet.
You can wrap your own content into blocks of data with your own headers or add other "synchronization" mechanisms to your system. Consider an HTTP server: it must wait until it reads an entire request like:
GET /index.html HTTP/1.1␍␊
Host: www.example.com␍␊
␍␊
Until the server sees the CRLFCRLF sequence, it must keep the partially-read data in a buffer. The bytes might come in one at a time in a dozen or more packets. Or, if the client is sending multiple requests in a single stream, a dozen requests might come in a single packet.
You just have to handle this.

Suggestions for developing a TCP/IP based message client

I've got a server side protocol that controls a telephony system, I've already implemented a client library that communicates with it which is in production now, however there are some problems with the system I have at the moment, so I am considering re-writing it.
My client library is currently written in Java but I am thinking of re-writing it in both C# and Java to allow for different clients to have access to the same back end.
The messages start with a keyword have a number of bytes of meta data and then some data. The messages are always terminated by an end of message character.
Communication is duplex between the client and the server usually taking the form of a request from the Client which provokes several responses from the server, but can be notifications.
The messages are marked as being on of:
C: Command
P: Pending (server is still handling the request)
D: Data data as a response to
R: Response
B: Busy (Server is too busy to handle response at the moment)
N: Notification
My current architecture has each message being parsed and a thread spawned to handle it, however I'm finding that some of the Notifications are processed out of order which is causing me some trouble as they have to be handled in the same order they arrive.
The duplex messages tend to take the following message format:
Client -> Server: Command
Server -> Client: Pending (Optional)
Server -> Client: Data (optional)
Server -> Client: Response (2nd entry in message data denotes whether this is an error or not)
I've been using the protocol for over a year and I've never seen the a busy message but that doesn't mean they don't happen.
The server can also send notifications to the client, and there are a few Response messages that are auto triggered by events on the server so they are sent without a corresponding Command being issued.
Some Notification Messages will arrive as part of sequence of messages, which are related for example:
NotificationName M00001
NotificationName M00001
NotificationName M00000
The string M0000X means that either there is more data to come or that this is the end of the messages.
At present the tcp client is fairly dumb it just spawns a thread that notifies an event on a subscriber that the message has been received, the event is specific to the message keyword and the type of message (So data,Responses and Notifications are handled separately) this works fairly effectively for Data and response messages, but falls over with the notification messages as they seem to arrive in rapid sequence and a race condition sometimes seems to cause the Message end to be processed before the ones that have the data are processed, leading to lost message data.
Given this really badly written description of how the system works how would you go about writing the client side transport code?
The meta data does not have a message number, and I have not control over the underlying protocol as it's provided by a vendor.

The requirement that messages must be processed in the order in which they're received almost forces a producer/consumer design, where the listener gets requests from the client, parses them, and then places the parsed request into a queue. A separate thread (the consumer) takes each message from the queue in order, processes it, and sends a response to the client.
Alternately, the consumer could put the result into a queue so that another thread (perhaps the listener thread?) can send the result to the client. In that case you'd have two producer/consumer relationships:
Listener -> event queue -> processing thread -> output queue -> output thread
In .NET, this kind of thing is pretty easy to implement using BlockingCollection to handle the queues. I don't know if there is something similar in Java.
The possibility of a multi-message request complicates things a little bit, as it seems like the listener will have to buffer messages until the last part of the request comes in before placing the entire thing into the queue.
To me, the beauty of the producer/consumer design is that it forces a hard separation between different parts of the program, making each much easier to debug and minimizing the possibility of shared state causing problems. The only slightly complicated part here is that you'll have to include the connection (socket or whatever) as part of the message that gets shared in the queues so that the output thread knows where to send the response.
It's not clear to me if you have to process all messages in the order they're received or if you just need to process messages for any particular client in the proper order. For example, if you have:
Client 1 message A
Client 1 message B
Client 2 message A
Is it okay to process the first message from Client 2 before you process the second message from Client 1? If so, then you can increase throughput by using what is logically multiple queues--one per client. Your "consumer" then becomes multiple threads. You just have to make sure that only one message per client is being processed at any time.

I would have one thread per client which does the parsing and processing. That way the processing would be in the order it is sent/arrives.
As you have stated, the tasks cannot be perform in parallel safely. performing the parsing and processing in different threads is likely to add as much overhead as you might save.
If your processing is relatively simple and doesn't depend on external systems, a single thread should be able to handle 1K to 20K messages per second.
Is there any other issues you would want to fix?

I can recommend only for Java-based solution.
I would use some already mature transport framework. By "some" I mean the only one I have worked with until now -- Apache MINA. However, it works and it's very flexible.
Regarding processing messages out-of-order -- for messages which must be produced in the order they were received you could build queues and put such messages into queues.
To limit number of queues, you could instantiate, say, 4 queues, and route incoming message to particular queue depending on the last 2 bits (indeces 0-3) of the hash of the ordering part of the message (for example, on the client_id contained in the message).
If you have more concrete questions, I can update my answer appropriately.