I'm trying to build multiple client - one server structure. Each client makes connection to server and server maintains those connection.
Client have two thread(thread1, thread2) running asynchronously and sharing 1 socket to send, receive. Two thread each contain multiple send, receive function. For server to serve each client, it also have to make two client handler that share 1 socket.
I wanna have my server to create client handler routine(asynchronous task) only when their(each client) socket is ready-to-read to save resource. I'm trying to implement this feature using 'Socket.Select' method.
Here's my assumed implementation of server side select routine using code close to c#.
server.selectRoutine(){
while(!serverSocket.closed())
{
checkReadList client_connect_socket_list_copy=new ArrayList<Socket>(client_connect_socket_list) ;
Socket.Select(client_connect_socket_list_copy, null,null) ;
foreach(Socket s in client_connect_socket_list_copy)
{
client_connect_info info=client_connect_info_dict.Items[s] ;
if(!info.is_client_thread1_handler_active)
{
info.is_client_thread1_active=true ;
clServerEntranceHandler handler=new clServerEntranceHandler() ;
Task.Run(handler.run()) ;
}
if(!info.is_client_thread2_handler_active)
{
info.is_client_thread2_active=true ;
clServerExitHandler handler=new clServerExitHandler() ;
Task.Run(handler.run()) ;
}
}
Code Explanation
Client_connect_socket_list: contains currently connected clients
client_connect_info_dict: client connect info dictionary mapped for each socket as key.
client_connect_info: consist of (is_client_thread1_Handler_Active, is_client_thread2_Handler_Active)
Select only those sockets that are ready to be read after Socket.Select
Run handler for those sockets where handler is currently not made, info.is_client_thread1_Handler_Active=false. And set info.is_client_thread1_Handler_Active=true. This is because there will be multiple recv function inside already-made handler and they are in blocking mode. So when client sends message to server destined to recv function blocked in already-made handler, Socket.Select will also think this socket as ready-to-read and it will end up creating new handler. To avoid this, i need to check whether handler is active. After handler finishes, it will set info.is_client_thread1_Handler_Active= false inside. Same for client_thread2 handler.
I think i now have to synchronize this routine with each handler already made. For example, avoid situation like recv function inside handler P snatch message A inside message queue of handler P's socket after Socket.Select saw message A inside message queue. Then handler P will end and set info.is_Handler_Active=false. Then selectRoutine will create new handler again for message A already read by handler P and there goes trouble.
I wanna know whether there is serious problem with this scheme that i should not use it. And if there is, then i wanna know better ways to implement multiple client - one server with room for scaling.
Related
I'm running into an interesting scenario when I'm trying to roll with .Net's SocketAsyncEventArgs. Namely, the fact that they can't seem to detect when a graceful remote socket shutdown has occurred.
Bit of background: I'm updating a legacy application from MFC to a .NET project, and my code needs to interface with all other legacy MFC code. In the legacy MFC code, the MFC backend automatically registers when a remote connection is gracefully closed with a FIN or RST signal. I've observed this behavior in action, and all the user can or needs to interact with is overloading the OnClose method that MFC provides.
I can't replicate that in C# or C++/CLI at the moment. My SocketAsyncEventArgs that I use to handle all receive operations looks like this:
static void AcceptHandler(System::IAsyncResult^ ar)
{
ServerSocket ^server = (ServerSocket ^)ar->AsyncState;
try
{
server->Socket = gcnew SocketMgr(server->listener->EndAcceptSocket(ar));
//pConnectionCb a function variable I use for updating the GUI when
//connection status changes. ReceiveDataHandler is another function
//variable for logging purposes.
if (server->pConnectionChangedCb)
{
server->pConnectionChangedCb(server->nID);
}
if (server->receiveDataHandler)
{
System::Net::Sockets::SocketAsyncEventArgs ^receiveArgs = gcnew System::Net::Sockets::SocketAsyncEventArgs();
receiveArgs->SetBuffer(server->readbuffer, server->nOffset, server->nBytesToGet - server->nOffset);
receiveArgs->Completed +=
gcnew System::EventHandler<System::Net::Sockets::SocketAsyncEventArgs ^>(server, &ServerSocket::IO_Completed);
server->Socket->ReceiveAsync(receiveArgs);
}
}
catch (System::Net::Sockets::SocketException ^e)
{
System::Windows::Forms::MessageBox::Show("OnAccept: Could not Accept, exception" + e->ErrorCode);
server->listener->EndAcceptSocket(ar);
}
}
void IO_Completed(System::Object ^sender, System::Net::Sockets::SocketAsyncEventArgs ^e)
{
if (!(e->SocketError == System::Net::Sockets::SocketError::Success))
{
kPrintf("Error.");
}
// determine which type of operation just completed and call the associated handler
switch (e->LastOperation)
{
case System::Net::Sockets::SocketAsyncOperation::Receive:
ProcessReceive(e);
break;
case System::Net::Sockets::SocketAsyncOperation::Send:
ProcessSend(e);
break;
default:
throw gcnew System::ArgumentException("The last operation completed on the socket was not a receive or send");
}
};
From what I've observed, when the remote socket ceases to exist, the SocketAsyncEventArgs object in the middle of the read exists in a state where it has not been completed, and will never be completed. As it fails to complete, IO_Completed will never be called, and I will be unable to use this to detect when a socket sends a graceful disconnect. So it can't be used.
...The only problem with this being, of course, that there's no OnRemoteClose (or equivalent) event for me to scribe to in Socket.Net.Sockets.Socket or in the SocketAsyncEventArgs, leaving me unable to detect a socket FIN or RST signal and keeping the socket open longer than expected. C# probably has a way around this, but I can't, for the life of me, find it. Anyone else wrestled with this before?
As it turns out, SocketAsyncEventArgs does record a graceful termination of any remote socket, regardless of client language. It does not expose the underlying TCP/IP events or anything similar, and instead just demonstrates the socket closure as an empty message sent.
My code, because of a PEBKAC error, was not receiving the empty 0-byte messages, and thus I could never 'see' the graceful shutdown.
(In the event anyone has this issue in the future, the problem is that the ProcessReceive method should have called ReceiveAsync to continue the loop after receiving its first signal, and it... wasn't, for reasons unrelated to the code.)
I'm attempting to learn ZeroMq for project at work although my background is in C#, and in the most simplest of tests I seem to have an issue where the socket.recv(...) call will block for the first received message, but after this throws an exception because the amount of data received is -1.
Currently my 'server' is:
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_REP);
socket.bind("tcp://127.0.0.1:5555");
while (true)
{
zmq::message_t message;
if (socket.recv(&message))
{
auto str = std::string(static_cast<char*>(message.data()), message.size());
printf("Receieved: %s\n", str.c_str());
}
}
This is basically from following the first example server within the ZeroMq documentation.
I'm pushing 1 bit of data from a C# 'client' using this code:
using (var context = new ZContext())
using (var requester = new ZSocket(context, ZSocketType.REQ))
{
requester.Connect(#"tcp://127.0.0.1:5555");
requester.Send(new ZFrame(#"hello"));
requester.Disconnect(#"tcp://127.0.0.1:5555");
}
Now I start the server, then start the client. I correctly receive the first message and I am correctly able to print this.
But now when I hit socket.recv(&message) again the code won't block but will instead throw an exception because the underlying zmq_msg_recv(...) returns a value of -1.
I'm unsure why this is occurring, I cannot see why it is expecting another message as I know that there is nothing else on this port. The only thing I came across is calling zmq_msg_close(...) but this should be called as part of the message_t destructor, which I have confirmed.
Is there anything I'm doing wrong in terms of the socket setup or how I'm using it for the recv(...) call to stop blocking?
Your problem is that you cannot receive 2 requests in a row with the REQ-REP pattern.
In the Request-Reply Pattern each request demands a reply. Your client needs to block until it receives a reply to its first request. Also, your server needs to reply to the requests before it services a new request.
Here is a quote referring to your exact issue from the guide.
The REQ-REP socket pair is in lockstep. The client issues zmq_send()
and then zmq_recv(), in a loop (or once if that's all it needs). Doing
any other sequence (e.g., sending two messages in a row) will result
in a return code of -1 from the send or recv call. Similarly, the
service issues zmq_recv() and then zmq_send() in that order, as often
as it needs to.
I have a network equipment to which I connect once using sockets, and the connection is maintained open all the time until application closes.
Now I have a class in C# that encapsulates the communication. There is a method SendMessage to the equipment. I need to use Socket.ReceiveAsync to get the response.
Let's say there are 3 methods: 1. GetEqValA(), GetEqValB(), GetEqValC() that call SendMessage with a specific message for the equipment.
I have created only one instance of socket Event args like that:
_completeArgs = new SocketAsyncEventArgs();
_completeArgs.SetBuffer(buffer, 0, buffer.Length);
_completeArgs.UserToken = _mySocket;
_completeArgs.RemoteEndPoint = _mySocket.RemoteEndPoint;
_completeArgs.Completed += new EventHandler<SocketAsyncEventArgs>(DataAvailable);
_mySocket.ReceiveAsync(_completeArgs);
Now, the DataAvailable method has something similar to the code below:
for (int i = 0; i < e.BytesTransferred; i++)
{
_tcpData.Add(e.Buffer[i]);
}
if (_tcpData.Count == _expectedTcpDataCount)
{
_expectedTcpDataCount = -1;
ProcessData();
// I don't want to put here, because it will wait for data until
// someone sends a message and the equipment responds with data
//_mySocket.ReceiveAsync(e);
}
else
{
_mySocket.ReceiveAsync(e);
}
Now, the 3 methods from above can be called by anyone, even different threads. I do have a lock mechanism for that.
My problem is that if I reuse _completeArgs in SendMessage for the next message to send, I get an exception that this eventArgs object is already in use by an asynchronous operation, whereas if I do the same(but not directly, by taking the SocketAsyncEventArgs e parameter from DataAvailable) in DataAvailable, no problem occurs.
_mySocket.ReceiveAsync(_completeArgs);
_mySocket.Send(pMessage);
The idea is that I don't want to call ReceiveAsync all the time, even if I know that nothing will come in there, but I want to call ReceiveAsync before sending any message to the device, because I know that I will get something.
The exception appears at method GetEqValC(), if I call them one after another in the sequence A,B,C.
What I don't understand, can you help me? Can I don what I want to do?
I use .NET 3.5.
P.S. Summary: I need to keep the connection alive, but read something from it only when I know for sure I must have something in there. Only one call at a time will be. One send, followed by one receive!
I am doing some basic Socket messaging. I have a routine that works well but there is a problem under load.
I'm using UDP to do a connectionless SendTo to basically do a ping-like operation to see if any of my listeners are out there on the LAN. Ideally I would just use the broadcast address, but Wireless routers don't seem to relay my broadcast. My work around is to iterate through all IPs on the Subnet and send my data gram to each IP. The other PCs are listening and if they get the message they will reply and that is how I get Peers to find each other. Here is the code that is in the loop which sends the data gram to each IP in the subnet.
string msgStr = "some message here...";
byte[] sendbuf = Encoding.ASCII.GetBytes(msgStr);
Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
socket.Blocking = true;
socket.SendTo(sendbuf, remoteEndPt);
//socket.Close();
This works, but when the Subnet range is large, say 255.255.0.0 (meaning ~60,000 IPs to iterate through) I will eventually get a SocketException with error code "10022", meaning "Invalid Argument". This tends to happen after ~10,000 or so successful sends then I start to see this error. Also, the router I use at work handles it and is presumably a high powered router, but the cheap-o one in my lab is the one that produces the error.
If I put in a wait time after catching the SocketException and before resuming the loop it will typically recover but eventually I'll get the error again.
I think what is happening is that the buffer on the router gets full and I cannot send anymore data. The higher quality one at work can handle it but the cheap-o one gets bogged down. Does that sound plausible?
A couple questions:
1) When using SendTo in a connectionless manner, do I need to call Close() on my Socket?
I've haven't seen any benefit in calling Close(), but when I do call Close() it severely slows down my iteration (I have it commented out above because it does slow things down a lot). Does this make sense?
2) Is there a way for me to tell I should wait before trying to send more data? It doesn't seem right to just catch the Exception which I still don't know what the cause of it is.
Thanks, J.
I am not sure that is the router only but I suspect that you are also running into some limit in the OS...
Any reason you are creating the Socket every time you send ?
Just reuse it...
Anyways according to http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.aspx it is a good idea to call Shutdown() and then Close() on the Socket... perhaps not with every send but every 255 IPs or so...
Checkout UdpClient - that could make implementation easier / more robust
EDIT - as per comment:
IF you want a Socket reuse "cache"... this for example would make sure that a specific Socket is only used every 256 checks...
// build/fill your Socket-Queue for example in the con
class SocketExample
{
Queue<Socket> a = new Queue<Socket>();
SocketExample ()
{
int ii = 0, C = 256;
for (ii = 0; ii < C; C++)
{
a.Enqueue (new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp));
}
}
// in your function you just dequeue a Socket and use it,
// after you are finished you enqueue it
void CheckNetIP (some parameters...)
{
Socket S = a.Dequeue();
// do whatever you want to do...
// IF there is no exception
a.Enqueue(S);
}
}
I read some C# chat source code & I see that: on chat server with a lot of connected clients, server listener will run in a separated thread & each connected client will also run in a separated thread.
Code examples:
Start server & begin listening in a separated thread:
public void StartListening()
{
// Get the IP of the first network device, however this can prove unreliable on certain configurations
IPAddress ipaLocal = ipAddress;
// Create the TCP listener object using the IP of the server and the specified port
tlsClient = new TcpListener(1986);
// Start the TCP listener and listen for connections
tlsClient.Start();
// The while loop will check for true in this before checking for connections
ServRunning = true;
// Start the new tread that hosts the listener
thrListener = new Thread(KeepListening);
thrListener.Start();
}
private void KeepListening()
{
// While the server is running
while (ServRunning == true)
{
// Accept a pending connection
tcpClient = tlsClient.AcceptTcpClient();
// Create a new instance of Connection
Connection newConnection = new Connection(tcpClient);
}
}
And a connection will also run in a separated thread:
public Connection(TcpClient tcpCon)
{
tcpClient = tcpCon;
// The thread that accepts the client and awaits messages
thrSender = new Thread(AcceptClient);
// The thread calls the AcceptClient() method
thrSender.Start();
}
So, if a chat server with 10000 connected clients, the chat server application will have 10002 threads (one main thread, one server thread & 10000 client threads). I think the chat server will be overhead with a big number of threads. Please help me a solution. Thanks.
UPDATE:
I believe chat examples are only for learning networking & they are not suitable in real-world model. Please give me a real-world solution. Thanks.
If you use .Net framework 2.0 SP2 or higher, than you may use new asyncrhronous sockets model based on IO Completion ports. In this case you shouldn't create your own threads, because IO Completion ports do all job for you.
Here some examples:
tcpServer = new System.Net.Sockets.TcpListener(IPAddress.Any, port);
tcpServer.Start();
tcpServer.BeginAcceptSocket(EndAcceptSocket, tcpServer);
private void EndAcceptSocket(IAsyncResult asyncResult)
{
TcpListener lister = (TcpListener)asyncResult.AsyncState;
Socket sock = lister.EndAcceptSocket(asyncResult);
//handle socket connection (you may add socket to you internal storage or something)
//start accepting another sockets
lister.BeginAcceptSocket(EndAcceptSocket, lister);
SocketAsyncEventArgs e = new SocketAsyncEventArgs();
e.Completed += ReceiveCompleted;
e.SetBuffer(new byte[socketBufferSize], 0, socketBufferSize);
sock.ReceiveAsync(e);
}
void ReceiveCompleted(object sender, SocketAsyncEventArgs e)
{
var sock = (Socket)sender;
if (!sock.Connected)
{
//handle socket disconnection
}
var buf = new byte[size];
Array.Copy(e.Buffer, buf, size);
//handle received data
//start reading new data
sock.ReceiveAsync(e);
}
A standard mechanism to ease the burden is known as selection, which can multiplex multiple Socket instances to watch for the ones that are ready to be read or written to. See this document: http://codeidol.com/csharp/csharp-network/Csharp-Network-Programming-Classes/Csharp-Socket-Programming/ and scroll down to the section on select().
1) You'll NEVER want that many threads running - even if you could get them to run on your box (which you can't - each thread has a stack associated with it that takes real RAM and as you start more and more and more you'll run out of physical resources in your box and watch it blow up).
2) You'll want to look into thread pooling - using a smaller amount of threads to tackle a larger amount of work - typically reading from a queue of work that you try to get through as quickly as possible.
3) You'll want to look into io completion ports - a means of having a callback when io (likek a disk read or a network io) is waiting for you to take action - think of a thread (or pool of threads) dedicated to getting io notifications and then shoving the action to take for that io into a queue and then another pool of threads that take care of the actual messaging/logging/etc.
4) What happens when you scale beyond one machine? Which you hope to do if you're successful right? :-) Typically people dedicate a set of N machines to chat - then they hash based on a identifier for the user (think a GUID that represented the user - or a UserID/bigint depending on what corresponds to some internal authentication token that is consistent from login to login) which allows them to deterministically route the user's status/state information to a specific machine in that set of N boxes dedicated to messaging. So if a user that hashes to server N[2] needs to check if theri friends ar logged in it is easy to know for each of their friends exactly which machine their friend's status should be in because the backend consistently hashes those friends to the IM machine that corresponds to each userid hash. (i.e. you know just from the userid what server in the farm should be handling the IM status for that user.
Just dont' think you're gonna spin up a bunch of threads and that will save the day. It's sloppy and works only in very small numbers.
To make the matter worse you would also have to communicate between some arbitrary number of threads (it's a chat server, people want to talk to each other, not themselves.) I would suggest looking into UDP - can be done with a single thread on the server and fits the network activity well - people rarely write more then couple of sentences at a time in chat exchanges, which is very convenient for size-limited UDP datagrams.
There are other approaches of course, but one sure thing though is that you will never be able to do thread per socket at that scale.
I suggest you to read this great article on MSDN Magazine.
Describing:
Threaded Server
Select-Based Server
Asynchronous Server
codes in C# & VB.Net