I'm working on a program which will asynchronously load large amounts of data from a server via gRPC (both ends use protobuf-net.Grpc).
While I can simply throw large amounts of data at the client via IAsyncEnumerable, I sometimes want to prioritize sending certain parts earlier (where the priorities are determined on the fly, and not know at the start, kind of like sending a video stream and skipping ahead).
If I were to send data and wait for a response every time I'd be leaving a lot of bandwidth unused. The alternative would be throwing tons of data at the client, which could cause network congestion and delay prioritized packets by an indefinite amount.
Can I somehow use HTTPS/2s / TCPs flow/congestion control for myself here? Or will I need to implement a basic flow/congestion control system on top of gRPC?
To be slightly more precise: I'd like to send as much data as possible without filling up any internal buffers, causing delays down the line.
Related
I am using a System.Net.Sockets.Socket in TCP mode to send data to some other machine. I implemented a listener to receive data formatted in a certain way and it is all working nicely. Now I am looking for ways to optimize use of bandwidth and delivery speed.
I wonder if it is worth sending a single array of bytes (or Span in the newer frameworks) instead of using a consecutive series of Send calls. I now send my encoded message in parts (head, length, tail, end, using separate Send calls).
Sure I could test this and I will but I do not have much experience with this and may be missing important considerations.
I realize there is a nowait option that may have an impact and that the socket and whatever is there lower in the stack may apply its own optimizing policy.
In would like to be able to prioritize delivery time (the time between the call to Send and reception at the other end) over bandwidth use for those messages to which it matters, and be more lenient with messages that are not time critical. But then again, I create a new socket whenever I find there is something in my queue and then use that until the queue is empty for what could be more than one message so this may not always work. Ideally I would want to be lenient so the socket can optimize payload until a time critical message hits the queue and then tell the socket to hurry until no more time critical messages are in the queue.
So my primary question is should I build my message before calling Send once (would that potentially do any good or just waste CPU cycles) and are there any caveats an experienced TCP programmer could make me aware of?
I have an application that needs to transfere many small (1-3Byte) messages via a TCP connection (WLAN).
I know that sending 100kB at once is much faster than sending them in very small pieces (nearly bytewise).
Nevertheless the information I need to transfere is only 1-3bytes in size. Collecting data before sending would increase throughput, but it is important that the small pieces of data are transfered as early/fast as possible. So gathering data before sending is the problem.
Now I ask, what would be the best way, not to send every message individually on the one hand and on the other hand not to delay their transmission longer than necessary.
Now I'm thinking about creating a little buffer. When the first message is add, I start a timer with 1ms timeout. After that millisecond, data will be transfered. Independend of how many bytes are in the queue. But I don't know if this is a good solution.
Isn't there a way that the TCPClient/Server classes of .NET themselves have a solution for such a problem. I mean, they should know, when the current transmission is finished. In the meantime they could accumulate all send requests and send them out as in one transaction.
The TCP stack by default will already buffer the data internally in order to reduce overhead when sending. See Nagle Algorithm for details. Just make sure that you have this algorithm enabled, i.e. set Nodelay to false.
I am wondering what approach to take
problem flow :
I have a web API which can accept request from the client
The API layer talks to business layer and then to data layer
Data layer Gets huge record set (5000000 rows) , now the business layer
process the columns and rows (using maximum threads of the processor)
once processed the API streams the content as an excel/csv to the client (browser)
Right now the entire download happens in one flow (fire and wait till response is ready)
I would like to isolate this huge business operation of processing 5000000 rows to a separate engine or task queue (I don't want my web site to fall to out of memory exception) , also I would like to make the user experience smooth.
Trying to use server push events/signalr/or browser long polling , so that I can push the file once the data/file is processed and ready .
Is there any better way to achieve the same?
Here are a few suggestions based on what I can understand
Serialization. I would not recommend to respond back with a CSV or Excel format for such a large dataset, unless it is the only format the client can handle. If you have some control over the workflow, I would change the client to accept a format such as JSON, or better yet go even more optimized serializers that are data and speed efficient like ProtoBuff, Avro, Thrift, etc.
Pagination. (Assuming you are able to implement the above suggestion.) Usually responding back with large data can hinder performance across the board. It is very common for an API to accept parameters to define the page number and page size. In your case, you could make a unique reference id for the query (ex. "query-001" and that can be called upon /api/query/001?page=10&items-per-page=10000).
Caching. If the query is made frequently, in order to reduce hitting your data layer with every query (for example, requesting different pages). You could either load the data onto the disk or keep the data in memory. Keeping a cache will significantly improve performance and could also reduce complex debugging problems when it comes to performance tuning the system.
I have two PCs connected by direct Ethernet cable over 1Gbps link. One of them act as TCP Server and other act as TCP Client/s. Now I would like to achieve maximum possible network throughput between these two.
Options I tried:
Creating multiple clients on PC-1 with different port numbers, connecting to the TCP Server. The reason for creating multiple clients is to increase the network throughput but here I have an issue.
I have a buffer Queue of Events to be sent to Server. There will be multiple messages with same Event Number. The server has to acquire all the messages then sort the messages based on the Event number. Each client now dequeues the message from Concurrent Queue and sends to the server. After sent, again the client repeats the same. I have put constraint on the client side that Event-2 will not be sent until all messaged labelled with Event-1 is sent. Hence, I see the sent Event order correct. And the TCP server continuously receives from all the clients.
Now lets come to the problem:
The server is receiving the data in little random manner, like I have shown in the image. The randomness between two successive events is getting worse after some time of acquisition. I can think of this random behaviour is due to parallel worker threads being executed for IO Completion call backs.
technology used: F# Socket Async with SocketEventArgs
Solution I tried: Instead of allowing receive from all the clients at server side, I tried to poll for the next available client with pending data then it ensured the correct order but its performance is not at all comparable to the earlier approach.
I want to receive in the same order/ nearly same order (but not non-deterministic randomness) as being sent from the clients. Is there any way I can preserve the order and also maintain the better throughput? What are the best ways to achieve nearly 100% network throughput over two PCs?
As others have pointed out in the comments, a single TCP connection is likely to give you the highest throughput, if it's TCP you want to use.
You can possibly achieve slightly (really marginally) higher throughput with UDP, but then you have the hassle of recreating all the goodies TCP gives you for free.
If you want bidirectional high volume high speed throughput (as opposed to high volume just one way at a time), then it's possible one connection for each direction is easier to cope with, but I don't have that much experience with it.
Design tips
You should keep the connection open. The client will need to ask "are you still there?" at regular intervals if no other communication goes on. (On second thought, I realize that the only purpose of this is to allow quick reponse and the possiblity for the server to initiate a message transaction. So I revise it to: keep the connection open for a full transaction at least.)
Also, you should split up large messages - messages over a certain size. Keep the number of bytes you send in each chunk to a maximum round hex number, typically 8K, 16K, 32K or 64K on a local network. Experiment with sizes. The suggested max sizes has been optimal since Windows 3 at least. You need some sort of protocol with a chunck consisting of a fixed header (typically a magic number for check and resynch, a chunk number also for check and for analysis, and a total packet length) followed by the data.
You can possibly further improve throughput with compression (usually low quick compression) - it depends very much on the data, and whether you're on a fast or slow network.
Then there's this hassle that one typically runs into - problems with the Nagle algorith - and I no longer remember enough of the details there. I believe I used to overcome that by sending an acknowledgement in return for each chunk sent, and I suspect by doing that you satisfy the design requirements, and so avoid waiting for the last bytes to come in. But do google this.
I'm adding file sharing capabilities to a basic .NET asynchronous socket server. The payload I'd like clients to send is just my headers+commandID+binaryFileData. This server needs to service requests from VB6 clients in addition to the .NET clients.
The party responsible for the VB6 clients worked out a convoluted way to transfer a file that I'm not particularly impressed by. It involves sending a small block of the file, at the end of which the server asks for the next block. This party claims that the VB6 Winsock control doesn't work well if you try to do a send with a large ("large" means anything not small -- 1MB is "large"). This sounds absurd to me.
I want clients to write a single large payload to the socket, and just do message reassembly/hashing at the server end. Are there actually problems with large writes in VB6 Winsock control, or is the other party making excuses?
No, Winsock and the socket controls have no concept of files or sizes, just a stream of bytes. I expect they're hitting the buffer size in which case they just need to send in chunks until it's all been sent. No need for the server to request the next chunk which will just slow things down.
The claim of VB6 having problems with large (for small values of large) payloads is absolutely true. Even more so, the resulting problems differ from installation to installation and by the phase of the moon. Sending more than a MB to a VB6 winsock control is asking for trouble. Been there, done that, just believe me.
That said, we went down another route: A function would accept a payload of arbitraty size, chunk it up in Megabytes and enqueue it. The winsock control events (SendComplete IIRC) would be used to dequeue the next Megabyte.
This is transparent to the consuming app (a singel call, indifferent of payload size), with the quirks worked around on the sending side - this works reliably without any sophisticated protocol, as the problems are completly inside the client.