I have a socket application that allows thousands of clients to connect. It stores them in a ConcurrentDictionary<int, Socket> and operates solely for request-response situations:
When I need data, I find the relevant socket and send a request, asking for the data I need.
After sending the request, I receive bytes until it sends the response. Then I stop receiving.
Like this:
public Task<Message> Request(int clientId, Message message)
{
Socket client;
return Clients.TryGetValue(clientId, out client)
? RequestInternal(client, message);
: _EmptyTask;
}
public async Task<Message> RequestInternal(Socket client, Message message)
{
await SendAsync(client, message).ConfigureAwait(false);
return await ReceiveOneAsync(client).ConfigureAwait(false);
}
Now I need to change this application to allow clients to send me anything, anytime; even without I make a request for it. Which -I think- will require constantly receiving from the sockets and a completely different approach.
Questions:
What are the bets-known approaches (best-practices) for this kind of applications?
Any gotchas you can tell me about or any guides you can point me to?
What I have in mind:
Disclaimer: This part is a little long and completely hypothetical. You can skip that if you have an answer to the above questions.
What I have in mind:
Receiving bytes constantly and adding the assembled PDUs to a BlockingCollection<Message>.
Creating a thread, dedicated to process received messages using BlockingCollection's GetConsumingEnumerable method.
The processing thread will do this:
foreach (var message in Messages.GetConsumingEnumerable())
ProcessMessage(message);
With this I can receive and process everything the clients send but distinguishing the messages that are sent to reply my requests from the messages that are sent because the client needed to would be an issue.
I think I can send a unique identifier byte (unique to that particular client) with the request. Then the client can send that identifier back to me in its response and I can use it to distinguish the response.
ProcessMessage(Message msg)
{
// msg is a message from msg.Sender.
if (msg.Id == 0)
{
// msg is not a response, do processing.
}
else
{
// msg is a response to the message that's sent with msg.Id.
// Find the request that:
// * ...is made to msg.Sender
// * ...and has the msg.Id as identifier.
// And process the response according to that.
}
}
This means I also have to store the requests.
Here is an hypothetical version of RequestInternal:
Edit: Replaced Wait calls with awaits after Stephen Cleary's answer.
private async Task RequestInternal(Socket client, Message message)
{
var request = new Request(client, message);
Requests.Add(request);
await SendAsync(client, message).ConfigureAwait(false);
return await request.Source.Task.ConfigureAwait(false);
}
And the Request class:
private sealed class Request
{
public readonly byte Id;
public readonly Socket Client;
public readonly Message Message;
public readonly TaskCompletionSource<Message> Source;
public Request(Socket client, Message message)
{
Client = client;
Message = message;
Source = new TaskCompletionSource<Message>();
// Obtain a byte unique to that socket...
Id = GetId(client);
}
}
And ProcessMessage becomes this:
ProcessMessage(Message msg)
{
if (msg.Id == 0)
OnReceived(msg); // To raise an event.
else
{
// Method to find a request using msg.Sender and msg.Id
var request = Requests.Find(msg);
if (request != null)
request.Source.SetResult(msg);
}
}
Although I have no idea what kind of collection type Requests would be.
Edit: I've used a ConcurrentDictionary<Key, Request> where Key is a private struct with an Int32 (id of the socket) and a Byte (id of the message) fields. it also implements IEquatable<T>.
I wrote a TCP/IP .NET Sockets FAQ a few years ago that addresses some common problems (such as message framing, continuous reading, and explanations of common errors). The code samples all use the Socket class, but the same concepts apply to all TCP/IP sockets.
Regarding your protocol design and request/response matching, the overall approach sounds good. You'll need to ensure you're threadsafe (e.g., Requests would probably be a ConcurrentDictionary). Also, you should await SendAsync rather than calling Wait.
An alternative approach that I've played around with but haven't put into production is based on TPL Dataflow. You can create one block that represents the "output" for each client and another block for the "input". Then you can layer your message framing on that, and layer your request/response matching on that, and then send any remaining (unsolicited) messages to a single shared BufferBlock.
So your "end-user" API would end up looking like this:
// Send a request and asynchronously receive a matching response.
Task<Message> RequestAsync(int clientId, Message message);
// Endpoint for unsolicited messages.
IReceivableSourceBlock<Tuple<int, Message>> UnsolicitedMessages { get; }
You could then hook up an ActionBlock to UnsolicitedMessages to execute a delegate whenever one comes in.
Related
My ASP.Net Core application provides a TCP listener, implemented with a custom ConnectionHandler, to receive binary data from another process (let's call it Datasource) on another host. This data is then sent to the browser through a WebSocket (called DataSink in the code).
Since the process Datasource has changed from a single TCP connection to UDP datagrams, I need to adapt (its internals are out of my reach).
How can I switch the current implementation to an UDP listener? Is there a canonical way how this is done with ASP.Net Core?
public class MySpecialConnectionHandler : ConnectionHandler
{
private readonly IMyDataSink DataSink;
public MySpecialConnectionHandler(IMyDataSink dataSink)
{
DataSink = dataSink;
}
public override async Task OnConnectedAsync(ConnectionContext context)
{
TransportConnection connection = context as TransportConnection;
Console.WriteLine("new connection: " + connection.RemoteAddress + ":" + connection.RemotePort);
while (true)
{
var result = await connection.Transport.Input.ReadAsync().ConfigureAwait(false);
var buffer = result.Buffer;
foreach (var segment in buffer)
{
await DataSink.RelayData(segment.Span.ToArray()).ConfigureAwait(false);
}
if (result.IsCompleted)
{
break;
}
connection.Transport.Input.AdvanceTo(buffer.End);
}
Console.WriteLine(connection.ConnectionId + " disconnected");
}
}
The UDP listener must be available while the ASP.Net Core application is running.
EDIT:
Order and reliability of the datagram transmission is not that important (perhaps not at all), since the transmitted data is a MPEG1-stream multiplexed into MPEG-TS. The data source is on the first host, the ASP.Net Core application is on a second host and the receiver / consumer is a third host. The host creating the stream and the receiving process on the third host are in separate networks. The ASP.Net Core application acts as a relay. The sender is sending all time, but does not care about whether the data is received or not.
EDIT 2:
The main problem right now is where to put the UdpClient. The previous implementation (back when we used TCP) configured the Kestrel server for additional TCP listening and used the already presented ConnectionHandler:
return WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.ConfigureKestrel((_, options) =>
{
// HTTP
options.Listen(networkInterface, httpPort);
// HTTPS
options.Listen(networkInterface, httpsPort, builder => builder.UseHttps());
// stream sink
options.Listen(streamInterface, streamPort, builder => builder.UseConnectionHandler<MySpecialConnectionHandler >());
});
The ConnectionHandler accepts the incoming TCP connection and then forwards the streaming data to a number of connected WebSockets. Since this is not usable with UDP datagrams, I need to place the UdpClient somewhere where it continuously (i.e. while(true)) receives datagrams and forwards them to the WebSockets. My problem is that I don't know where to put it, run the background thread and have the communication span threads without having any problems with this inter-thread data flow (like race conditions).
So, to conclude this:
We used a combination of a BackgroundWorker with an UdpClient. The BackgroundWorker is only instantiated when there is at least one receiver:
StreamReceiver = new BackgroundWorker();
StreamReceiver.DoWork += ReceiveStream;
StreamReceiver.RunWorkerAsync();
ReceiveStream is a private method that establishes the UdpClient and then waits for incoming data that needs to be relayed.
private async void ReceiveStream(object sender, DoWorkEventArgs e)
{
// DataSinkPort is a const int
UdpClient datasource = new UdpClient(_DataSinkPort);
while (true)
{
var rec = await datasource.ReceiveAsync();
await RelayData(rec.Buffer);
if (_CancellationToken.IsCancellationRequested)
{
return;
}
}
}
The method RelayData just uses the outgoing TCP connection of each subscribed receiver.
hello all i have this issue , and i am stuck with it so any help will be greatly appreciated
i have to build a socket chat (client Server) module and i have done almost 80% of the work but now i am stuck , Scenario is that i have a server app and clients connect to it now if say 4 clients are connected to server each of them can communicate to each other , one client will send message and server will receive that message and will pass it along ,this is working very fine but when 2 or more clients send a message to 3rd client at the same time than i can not get one message and client get disconnected i know i have to build up a queue structure but i am not getting succeeded in it here
here is my code structure
StartReceive(SocketAsyncEventArgs) ->
First of all i call this method to start listening for incoming messages
and when i got one message i check if it is completed Sync or Async
and after this there is an io completed listener which is called every time one receive opertaion is completed and in that io completed method i do call processReceive method which is used to process received chunks
so finally my code structure is like this
StartReceive-> IO Completed -> ProcessReceive
i have designed the structure so that every client will have a Receive SOCKETASYNCEVENTAGRS
object and a Send SOCKETASYNCEVENTAGRS at server side and i do maintain a pool for this
so each client receives and sends data through its own SOCKETASYNCEVENTAGRS object
i want a scenario such that if two or more clients send messages to 3rd client than 3rd client should not receive all messages at the same time instead it should have a queue structure and it should receive one message at a time and same for send operation
here is some of my code
private void StartReceive(SocketAsyncEventArgs receiveSendEventArgs)
{
DataHoldingUserToken receiveSendToken = (DataHoldingUserToken)receiveSendEventArgs.UserToken;
receiveSendEventArgs.SetBuffer(receiveSendToken.bufferOffsetReceive, this.socketListenerSettings.BufferSize);
bool willRaiseEvent = receiveSendEventArgs.AcceptSocket.ReceiveAsync(receiveSendEventArgs);
if (!willRaiseEvent)
{
ProcessReceive(receiveSendEventArgs);
}
}
This is IO Completed
void IO_Completed(object sender, SocketAsyncEventArgs e)
{
DataHoldingUserToken receiveSendToken = (DataHoldingUserToken)e.UserToken;
switch (e.LastOperation)
{
case SocketAsyncOperation.Receive:
ProcessReceive(e);
break;
default:
}
}
and this is my StartReceive with Queue implemented but it is not working
Queue rec = new Queue();
bool IsExecuted = true;
private void StartReceive(SocketAsyncEventArgs receiveSendEventArgs)
{
if (IsExecuted)
{
DataHoldingUserToken receiveSendToken = (DataHoldingUserToken)receiveSendEventArgs.UserToken;
rec.Enqueue(receiveSendToken);
}
try
{
receiveSendEventArgs.SetBuffer(receiveSendToken.bufferOffsetReceive, this.socketListenerSettings.BufferSize);
bool willRaiseEvent = receiveSendEventArgs.AcceptSocket.ReceiveAsync(receiveSendEventArgs);
if (!willRaiseEvent)
{
ProcessReceive(receiveSendEventArgs);
}
rec.Dequeue();
IsExecuted = true;
}
catch
{
IsExecuted = false;
StartReceive(receiveSendEventArgs);
}
}
looking for a decent help or some good direction
I'm not sure if this is the root cause of your problem but it seems to me that you are running out of DataHoldingUserToken at some time because you are never doing anything with the objects you dequeue, hence you are throwing it away.
Also note that it is recommended to use the generic form of the Queue class for non-trivial object. That would be System.Collections.Generic.Queue<DataHoldingUserToken>.
I've got a multithreaded Windows service which is consuming messages off a Rabbit queue and sending emails based on the content of the messages.
When the rabbit client is initialized at startup, it limits the Threadpool threads with a Min and Max value.
For each message taken off the queue the service is sending an HTTP request to a web api service using HttpClient GetAsync method to retrieve email address.
The problem is that the request goes off to the data service, but the response never comes back. The windows service keeps consuming messages off the queue and hangs after a while (probably runs of of free threads) - it's waiting for any of the calls to web api to complete which they never do.
I was able to resolve the problem using a Semaphore class for the Rabbit loop rather than trying to limit the Threadpool directly, however, I'd like to know why the service got into this state in the first place. Is that to do with the GetAsync call? Is it perhaps freeing up the thread for the duration of the request, so that the main loop can steal it for a next request?
Any ideas?
The original loop:
while (!_stopped)
{
if (_paused) continue;
try
{
using (var messageBusReceiver = _rabbitQueueClient.ConfigureMessageBusReceiver())
{
using (_consumer = messageBusReceiver.Listen<PublishableItem>())
{
while (!_stopped)
{
if (_paused) continue;
_consumer.Consume(callback, consumeSynchronously: false);
_communicationErrorCount = 0;
}
}
}
}
The Consume method is eventually doing this:
_threadPoolProvider.QueueUserWorkItem(o =>
consumeMessage(callback, eventArgs, o), message);
The callback begins with the following lines - the null checking line is never reached:
var foo = _fooService.GetFoo(messageInfo.FooId);
if (foo == null)
{
throw new FooNotFoundException(
String.Format(CultureInfo.InvariantCulture, "Foo was not found for FooId of {0}", messageInfo.FooId));
}
The client method:
public Foo GetFoo(Guid id)
{
var path = getPathWithQueryStringAndDebug("getfoo", "id", id.ToString());
var response = _client.GetAsync(path).Result;
return processResponse<FooDto>(response);
}
I have a streaming server that with a contract looking something like this:
[ServiceContract]
public interface IStreamingService
{
[OperationContract(Action = "StreamingMessageRequest", ReplyAction = "StreamingMessageReply")]
Message GetStreamingData(Message query);
}
Here's a rudimentary implementation with stuff (like error handling) removed to simplify things:
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
[ErrorBehavior(typeof(StreamingServiceErrorHandler))]
public class StreamingService : IStreamingService
{
public StreamingService()
{
}
public Message GetStreamingData(Message query)
{
var dataQuery = query.GetBody<DataQuery>();
// Hook up events to let us know if the client disconnects so that we can stop the query...
EventHandler closeAction = (sender, ea) =>
{
dataQuery.Stop();
};
OperationContext.Current.Channel.Faulted += closeAction;
OperationContext.Current.Channel.Closed += closeAction;
Message streamingMessage = Message.CreateMessage(
MessageVersion.Soap12WSAddressing10,
"QueryMessageReply",
new StreamingBodyWriter(QueryMethod(dataQuery));
return streamingMessage;
}
public IEnumerable<object> QueryMethod (DataQuery query)
{
// Returns a potentially infinite stream of objects in response to the query
}
}
This implementation uses a custom BodyWriter to stream results from the QueryMethod:
public class StreamingBodyWriter : BodyWriter
{
public StreamingBodyWriter(IEnumerable items)
: base(false) // False should be passed here to avoid buffering the message
{
Items = items;
}
internal IEnumerable Items { get; private set; }
private void SerializeObject(XmlDictionaryWriter writer, object item)
{
// Serialize the object to the stream
}
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
foreach (object item in Items)
{
SerializeObject(writer, item);
}
}
}
The client connects and starts reading the data stream. Something like this:
public IEnumerable<T> GetStreamingData<T>(Message queryMessage)
{
Message reply = _Server.GetStreamingData(queryMessage);
// Get a chunckable reader for the streaming reply
XmlReader reader = reply.GetReaderAtBodyContents();
// Read the stream of objects, deserializing each one as appropriate
while (!reader.EOF)
{
object item = DeserializeObject(reader);
if (item == null)
continue;
// Yield each item as it's deserialized, resulting in a [potentially] never-ending stream of objects
yield return (T)item;
}
}
This works very well and I get a stream of objects back to the client. Very nice. The problem is when the client disconnects mid-stream (either gracefully or ungracefully). In either case, the server doesn't get notified of this disconnect other than as a fault picked up by the service's error handler.
As you can see, I've tried hooking the Faulted and Closed channel events in the service method but they are not firing when the client disconnects.
I suspect that these events don't fire because with a streaming contract the service has already returned from the operation method (GetStreamingData). This method has returned a Message with a custom body-writer and it's this body-write that (lower in the service channel stack) is grabbing results from the calculation thread (via an IEnumerable) and streaming them in to the reply message. Since the operation method has returned then I'm guessing these channel events don't get fire.
The binding is a customized version of net.tcp binding (non-duplex) with only two binding elements: BinaryMessageEncodingBindingElement and TcpTransportBindingElement. No security or anything. Basically just raw net.tcp with binary messages.
The problem is that when a client disconnects, I want the server to stop the background calculation thread that is generating results and feeding them into the IEnumerable being read by the BodyWriter. The body writer has stopped (obviously), but the thread continues to live.
So where can I hook to discover if a client has disconnected mid-stream?
Update:
There are two main disconnection cases: Explicit disconnection, due either to the disposal of the client proxy or termination of the process at the client; Passive disconnection, usually due to a network failure (example: ISP drops the connection, or the network cable gets yanked).
In the first case, there is an explicit connection exception that get's received by the server when it tries to send new data down the stream. No problem.
The second scenario of a broken network pipe is more troublesome. Usually this condition is detected based on the Send timeout, but since this is a streaming interface the send timeout is cranked way up (I have it cranked to several days since it is conceivable that streaming data transfers could last that long). So when a client is disconnected due to a flaky network pipe, the service continues to send data down a connection that does really exist any more.
At this point I don't know of a good way to resolve the second option. Suggestions?
The Faulted event is really the best thing for handling these types of scenarios, but it will only fire if reliableSession is enabled. It is enabled by default in the standard netTcpBinding, but won't be enabled in this case because you are using a custom stripped down version of the binding. Try adding this to your custom binding :)
I'm struggling a bit with socket programming (something I'm not at all familiar with) and I can't find anything which helps from google or MSDN (awful). Apologies for the length of this.
Basically I have an existing service which recieves and responds to requests over UDP. I can't change this at all.
I also have a client within my webapp which dispatches and listens for responses to that service. The existing client I've been given is a singleton which creates a socket and an array of response slots, and then creates a background thread with an infinite looping method that makes "sock.Receive()" calls and pushes the data received into the slot array. All kinds of things about this seem wrong to me and the infinite thread breaks my unit testing so I'm trying to replace this service with one which makes it's it's send/receives asynchronously instead.
Point 1: Is this the right approach? I want a non-blocking, scalable, thread-safe service.
My first attempt is roughly like this, which sort of worked but the data I got back was always shorter than expected (i.e. the buffer did not have the number of bytes requested) and seemed to throw exceptions when processed.
private Socket MyPreConfiguredSocket;
public object Query()
{
//build a request
this.MyPreConfiguredSocket.SendTo(MYREQUEST, packet.Length, SocketFlags.Multicast, this._target);
IAsyncResult h = this._sock.BeginReceiveFrom(response, 0, BUFFER_SIZE, SocketFlags.None, ref this._target, new AsyncCallback(ARecieve), this._sock);
if (!h.AsyncWaitHandle.WaitOne(TIMEOUT)) { throw new Exception("Timed out"); }
//process response data (always shortened)
}
private void ARecieve (IAsyncResult result)
{
int bytesreceived = (result as Socket).EndReceiveFrom(result, ref this._target);
}
My second attempt was based on more google trawling and this recursive pattern I frequently saw, but this version always times out! It never gets to ARecieve.
public object Query()
{
//build a request
this.MyPreConfiguredSocket.SendTo(MYREQUEST, packet.Length, SocketFlags.Multicast, this._target);
State s = new State(this.MyPreConfiguredSocket);
this.MyPreConfiguredSocket.BeginReceiveFrom(s.Buffer, 0, BUFFER_SIZE, SocketFlags.None, ref this._target, new AsyncCallback(ARecieve), s);
if (!s.Flag.WaitOne(10000)) { throw new Exception("Timed out"); } //always thrown
//process response data
}
private void ARecieve (IAsyncResult result)
{
//never gets here!
State s = (result as State);
int bytesreceived = s.Sock.EndReceiveFrom(result, ref this._target);
if (bytesreceived > 0)
{
s.Received += bytesreceived;
this._sock.BeginReceiveFrom(s.Buffer, s.Received, BUFFER_SIZE, SocketFlags.None, ref this._target, new AsyncCallback(ARecieve), s);
}
else
{
s.Flag.Set();
}
}
private class State
{
public State(Socket sock)
{
this._sock = sock;
this._buffer = new byte[BUFFER_SIZE];
this._buffer.Initialize();
}
public Socket Sock;
public byte[] Buffer;
public ManualResetEvent Flag = new ManualResetEvent(false);
public int Received = 0;
}
Point 2: So clearly I'm getting something quite wrong.
Point 3: I'm not sure if I'm going about this right. How does the data coming from the remote service even get to the right listening thread? Do I need to create a socket per request?
Out of my comfort zone here. Need help.
Not the solution for you, just a suggestion - come up with the simplest code that works peeling off all the threading/events/etc. From there start adding needed, and only needed, complexity. My experience always was that in the process I'd find what I was doing wrong.
So is your program SUDO outline as follows?
Socket MySocket;
Socket ResponceSocket;
byte[] Request;
byte[] Responce;
public byte[] GetUDPResponce()
{
this.MySocket.Send(Request).To(ResponceSocket);
this.MySocket.Receive(Responce).From(ResponceSocket);
return Responce;
}
ill try help!
The second code post is the one we can work with and the way forward.
But you are right! the documentation is not the best.
Do you know for sure that you get a response to the message you send? Remove the asynchronous behavior from the socket and just try to send and receive synchronously (even though this may block your thread for now). Once you know this behavior is working, edit your question and post that code, and I'll help you with the threading model. Once networking portion, i.e., the send/receive, is working, the threading model is pretty straightforward.
One POSSIBLE issue is if your send operation goes to the server, and it responds before windows sets up the asynchronous listener. If you arent listening the data wont be accepted on your side (unlike TCP)
Try calling beginread before the send operation.