So I have written this client/server socket application that uses the SocketAsyncEventArgs "method" for doing async sockets.
Using the same library I have used for many other applications, I now for the first time experience a situation that I never anticipated.
Our new client/server application when started, starts to send lot's of data in both directions.
When done in unit-tests using mock-objects (without delays) to mimic normal socket operations, it all works well.
But in real situations using real sockets, we get a sort of deadlock where both endpoints are stuck in a Socket.SendAsync() operation (yes it returned true, was not synchronously handled)
My idea is that the receive buffer of both parties are full, and the tcp stack is not acknowleding any frames anymore. (connected to 127.0.0.1)
So I made the receivebuffer twice as large as the sendbuffer, but unfortunately it is not that simple due to the nature of our "protocol", and how we determine to send or receive.
I now have to re-think the method that determines when to start sending and when to start receiving.
A complicating factor is, that the purpose of this connection is to mutliplex multiple bi-directional general purpose communication channels over this socket connection. That means that there is no pre-determined sequence of communication, all channels may have their own protocols.
Of course, there is a tls initiation, handshake and authentication, which all work well, but when the connection becomes operational, and the channels start their own communications, the only sure thing is that received data has a size and channelnumber as a header.
After each operation, I check to see if there is any waiting data in the receivebuffer, or by checking Socket.Available.
This combined with measuring how much data was received since last sent operation, and how full the transmitbuffer is getting, I decide to receive more or start sending, or do nothing, and poll again in xx ms.
I now realize that this is wronge.
Am I trying to accomplish something that is simply not possible using only one socket connection?
Anyone every tried to accomplish something simular, or know a good way of accomplish a safe way that does not introduce these odd lock-ups.
Thanks,
Theo.
Related
NOTE: This is one-way UDP transmission (i.e. no return path)--I CANNOT send data back to the client.
At this point, I've written a few client/server combinations all with a mix of good and bad results for packet delivery on the server-side using the .NET UDPClient class. Here is a synopsis of them:
Single, blocking thread/single port (.Receive()): Works well except unless I slow down transmission of data, I inevitably lose a small amount of packets which makes the resulting file unusable.
Single, async thread/single port (.BeginReceive()): Same as above -- I miss packets if I don't throttle back the sender (client).
Multi-thread, multi-ports: I create 100 threads (random number) and set a static port range (e.g. 10001-10100). Had to change the client to dynamically walk through sending across different ports. This seems to work well in initial testing but presented new problems (e.g. file locking issues trying to log across the different threads).
At this point, since I'm using UDP and can't get missed packets re-transmitted, I'm leaning towards option 3. For option 3, does anyone see additional problems of concern (besides the access-contention issues I'm already experiencing)?
Is anyone aware of any other strategy to employ for the server to minimize the possibility of losing packets to a non-ready UDP listener?
I'm programming a Socket/Client/Server library for C#, since I do a lot of cross-platform programming, and I didn't find mono/dotnet/dotnet core enough efficient in high-performance socket handling.
Sice linux epoll unarguably won the performance and usability "fight", I decided to use an epoll-like interface as common API, so I'm trying to emulate it on Windows environment (windows socket performance is not that important for me than linux, but the interface is). To achieve this I use the Winsock2 and Kernel32 API's directly with Marshaling and I use IOCP.
Almost everything works fine, except one: When I create a TCP server with winsock, and connect to it (from local or from a remote machine via LAN, does not matter) with more than 10000 connections, all connections are accepted, no problem at all, when all connections send data (flood) from the client side to the server, no problem at all, server receives all packets, but when I disconnect all clients at the same time, the server does not recognize all disconnection events (i.e. 0 byte read/available), usually 1500-8000 clients stuck. The completion event does not get triggered, therefore I can not detect the connection loss.
The server does not crash, it continues accept new connections, and everything works as expected, only the lost connections do not get recognized.
I've read that - because using overlapped IO needs pre-allocated read buffer - IOCP on reading locks these buffers and releases the locks on completion, and if too many events happen in the same time it can not lock all affected buffers because of an OS limit, and this causes IOCP hang for indefinite time.
I've read that the solution to this buffer-lock problem is I should use a zero-sized buffer with null-pointer to the buffer itself, so the read event will not lock it, and I should use real buffer only when I read real data.
I've implemented the above workaround and it works, except the original problem, after disconnecting many-thousands of clients in the same time, a few-thousand stuck.
Of course I keep up the possibility my code is wrong, so I made a basic server with dotnet's built in SocketAsyncEventArgs class (as the official example describes), that basically does the same using IOCP, and the results are the same.
Everything works fine, except the thousands of client disconnecting in the same time, a few-thousand of disconnection (read on disconnect) events does not get recognized.
I know I should do IO operation and check the return value if the socket is still can perform the IO, and if not, then disconnect it. The problem is in some cases I have nothing to tell the socket, I just receive data, or if I do it periodically this would be almost the same as polling, and would cause high load with thousands of connections, wasted CPU work.
(I use closing the clients numerous closing methods, from gaceful disconnection to proper TCP Socket closing, both on windows and linux clients, results are always the same)
My questions:
Is there any known solution to this problem?
Is there any efficient way to recognize TCP (graceful) connection closing by remote?
Can I somehow set a read-timeout to overlapped socket read?
Any help would be appreciated, thank You!
I am currently on a personal project for learning purposes. I want to make a connection over UDP, for application such as games. Each datagram sent has a specific header that indicates which "logic" channel it belongs to - for example, channel 0 is just like UDP with the extra header overhead, and channel 1 uses more headers to bring some extra reliability. The channels objective is to "automatically" separate messages into logic groups, up to a specific amount.
In my current code, there is a simple loop in a separate thread that handles sending and receiving:
// This is pseudo code
public void Tick() {
if(Socket.Poll) {
do {
ReadMessage();
} while(Socket.Available > 0)
}
SendQueuedOutgoingMessages();
}
Though this works on an ideal world, I have this feeling that this logic fails when there are too many incoming or outgoing messages. Is it possible to use the same socket to simultaneously send and receive messages (i.e send and receive are asynchronous or in different threads)? Even if it is possible, would it be better if I simply used two or more UDP sockets (or mix TCP and UDP sockets, if I need reliability), having specially maintainability in mind?
The most direct alternative I can think around this would be to use a scheduling algorithm to control how many messages to read and send, by means of queue sizes or other factors, but this feels poor and inflexible in this situation.
Edit: Adding more information about the code.
The Tick() method is set to be called a specific amount of times per second if it immediately returns. For example, 30 times per second if no new in or out messages exists, and less if it needs some time to send or receive data. I've used blocking ReceiveFrom and SendTo methods as to avoid busy waiting or calls such as Sleep(0).
Though I immediately treat incoming messages, I use an outgoing messages queue to help with the channels idea - each channel has its own priority down to 'no priority', affecting its bandwidth share over time in smooth and busy moments.
Whether or not using 2 sockets for receiving and sending separately, or just a single one depends on the situation. If you are going to send a lot of messages, and even if you are on a dedicated thread, the socket might block if the outgoing queue that is used by the socket becomes full.
There are several solutions to this problem. Using 2 sockets and 2 distinct threads is one of them, using select in combination with asynchronous sockets is another. The point is that you don't want to stop receiving just because a send might block.
Each of these possible solutions have their own complexity.
The select api is meant to check if for a certain socket there is something to receive, but you can also use it to detect if a socket becomes writable again. You need a socket option to put the socket in non blocking state and you need to check for E_WOULDBLOCK return codes with each send. If so the send has failed and you have to queue the message yourself.
You don't really send and receive at the same time, it is sequentially. You use select to check if a socket is writable and readible by using 2 bitmasks manipulated with the fd_set api. You can use select even on multiple sockets at once. Then if select, which is a blocking call, returns, you can check each individual bit to check what actions needs to be performed.
The reason why a send can block, if a socket is not put in nonblocking state is that the output queue of the socket can be full. If the socket is blocking, it would simply wait for the queue to become ready again. But during that wait, you cannot receive anything on that same socket. This is the very reason why you need non blocking sockets and the select api, in combination with some kind of queueing mechanism yourself.
Why not simply the standard read loop setup?
while (true)
ReadMessage();
There is no scheduling or throttling necessary. It is not necessary to know whether a packet is ready or not.
You can read and write simultaneously on the same UDP socket.
There is no need for an outgoing queue, either. Just send.
I have the requirement of receiving a stream of UDP packets at a fixed rate of 1000 packets per second. They are composed of 28 bytes of payload, where the first four bytes (Uint32) are the packet sequence number. The sender is an embedded device on the same local network, and addresses and ports are mutually known. I am not expected to receive every packet, but the host should be designed so that it doesn't add to the intrinsic UDP protocol limitations.
I have only limited previous experience with UDP and Sockets in general (just casual streaming from sensor app on android phone to PC, for a small demo project).
I am not sure what is the most sensible way to receive all the packets at a fast rate. I imagine some sort of loop, or else some sort of self-retriggering receive operation. I have studied the difference between synchronous and asynchronous receives, and also the Task-based Asynchronous Pattern (TAP), but I don't feel very confident yet. It looks like a simple task, but its understanding is still complicated to me.
So the questions are:
Is UdpClient class suitable to a scenario like this, or am I better off going with the Socket class? (or another one, by the way)
Should I use synchronous or asynchronous receiving? If I use synchronous, I am afraid of losing packets, and if I use asynchronous, how should I clock/throttle the receive operations so that I don't start them at a rate that's too large?
I thought about a tight loop testing for UdpClient.Available() like below. Would it be a good design choice?
while (running)
{
if (udpSocket.Available() > 0)
{
// do something (what?)
}
}
Just read from your UdpClient as fast as you can (or as fast as is convenient for your application) using UdpClient.Receive(). The Receive() operation will block (wait) if there is no incoming packet, so calling Receive() three times in a row will always return you three UDP packets.
Generally, when receiving UDP packets you do not have to worry about the sending rate. Only the sender needs to worry about the sending rate. So the opposite advice for the sender is: Do not send a lot of UDP packets as fast as you can.
So my answers would be:
Yes, use UdpClient.
Use synchronous receiving. Call UdpClient.Receive() in a loop running in your receive thread, of in the main loop of your program, what ever you like.
You do not need to check for available data. UdpClient.Receive() will block until there is data available.
Do not be afraid of loosing packets when receiving UDP: You will almost never lose UDP packets on the receiving host because you do something wrong in receiving. UDP packets mostly get dropped by network components like routers somewhere on the network path, which cannot send (forward) UDP packets as fast as they receive them.
Once the packets arrive at your machine, your OS does a fair amount of buffering (for example 128k on Linux), so even if your application is unresponsive for say 1 second it will not lose any UDP packets.
I'm writing an instant messaging server in C# for learning purposes.
My question is whether I should use synchronous or asynchronous sockets to handle the IM clients. The goal is to handle as many clients as possible.
I'm not quite sure but as far as I know with async sockets the packets don't arrive in order which means when you send 2 chat messages and there is a delay/lag it's possible that the second one arrive before the first one. Is this right and if so, is there a way to solve this issue?
About sync sockets: Is synchronous sockets a good solution for many clients? Do I have to check every socket/connection in a loop if there are new packets? If so, isn't this quite slow?
Last question: Assume I want to implement a way to send files (e.g. images) through the protocol (which is a non-standard binary protocol btw), can I still send messages while uploading?
The goal is to handle as many clients as possible.
Async then. It scales a lot better.
I'm not quite sure but as far as I know with async sockets the packets don't arrive in order which means when you send 2 chat messages and there is a delay/lag it's possible that the second one arrive before the first one.
TCP guarantees that everything arrives in order.
Assume I want to implement a way to send files (e.g. images) through the protocol (which is a non-standard binary protocol btw), can I still send messages while uploading
I recommend that you use a separate connection for file transfers. Use the first connection to do a handshake (determine which port to use and specify file name etc). Then use Socket.SendFile on the new socket to transfer the file.
Everything #jgauffin said (i.e. TCP handles packet-order, async better for n(clients) > 1000).
Assume I want to implement a way to send files (e.g. images) through the protocol (which is a non-standard binary protocol btw), can I still send messages while uploading?
Your custom protocol has to be built to support this. If you write a 8MB packet to the Socket, you won't be able to write anything else using that socket until the 8MB are sent. Instead, use upload-chunks of smaller size so that other packets have the chance to go over the pipe as well.
[UPLOAD id=123 START length=8012389]
[UPLOAD id=123 PART chunk=1 length=2048 data=...]
[UPLOAD id=123 PART chunk=2 length=2048 data=...]
[MESSAGE to="foo#example.com" text="Hi"]
[UPLOAD id=123 PART chunk=3 length=2048 data=...]
// ...
[UPLOAD id=123 COMPLETE checksum=0xdeadbeef]
The difference between an async approach and a sync approach is more about the difference between non-blocking and blocking io. With both approaches, the data is delivered in the same order that it has been transmitted. However, you don't block while you wait for an async call to complete, so you can start transmitting to all of your clients, before any of the individual communications has finished writing to the socket (which is why typically it would be the approach followed by servers).
If you go down the sync route, you block until each transmission / receive operation has completed, which means you may require need to run multiple threads to handle the clients.
As far as uploading an image at the same time as sending messages, you may want to handle that down a different pipe connection between the client/server so that it doesn't cause a blockage.