Develop Reference

Couldn't fetch response more than 4096 byte in Chat (Socket) [duplicate]

I am receiving JSON through a websocket. At least: I am partially. Using an online websocket service I receive the full JSON response (all the HTML markup is ignored). When I look at the JSON that I receive in my console I can see the HTML markup (viewing it with the HTML viewer during debugging removes the HTML) but it ends abruptly (incomplete data).
My buffer has plenty of space and I am using async-await to (supposedly) wait for the entire response to come in before continuing.
private async Task Receive()
{
var buffer = new byte[4096 * 20];
while (_socket.State == WebSocketState.Open)
{
var response = await _socket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
if (response.MessageType == WebSocketMessageType.Close)
{
await
_socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Close response received",
CancellationToken.None);
}
else
{
var result = Encoding.UTF8.GetString(buffer);
var a = buffer[1000];
var b = buffer[10000];
var c = buffer[50000];
var d = buffer[81000];
Console.WriteLine(result);
var responseObject = JsonConvert.DeserializeObject<Response>(result, _requestParameters.ResponseDataType);
OnSocketReceive.Invoke(this, new SocketEventArgs {Response = responseObject });
buffer = new byte[4096 * 20];
}
}
}
Things to note: The buffer is perfectly big enough and b, c and d are never filled. I should also note that this only happens for the 1-questions-newest-tag-java request, 155-questions-active works perfectly fine.
After doing some digging I have found that response.CloseStatus and response.CloseStatusDescription are always null, response.Count is always 1396 (copy-pasting the result in Word does show that there are always 1396 characters) and response.EndOfMessage is false.
Digging through some source code I have found that the DefaultReceiveBufferSize is 16 * 1024 (big enough) and the WebSocketGetDefaultKeepAliveInterval() refers to an external implementation (but the debugger shows 00:00:30).
It is not a matter of timeout since the debugger halts at the same moment the online service receives its response.
Why is my method continuing to execute when the socket has not yet received all data?

Just to complete #Noseratio response, the code would be something like this:
ArraySegment<Byte> buffer = new ArraySegment<byte>(new Byte[8192]);
WebSocketReceiveResult result= null;
using (var ms = new MemoryStream())
{
do
{
result = await socket.ReceiveAsync(buffer, CancellationToken.None);
ms.Write(buffer.Array, buffer.Offset, result.Count);
}
while (!result.EndOfMessage);
ms.Seek(0, SeekOrigin.Begin);
if (result.MessageType == WebSocketMessageType.Text)
{
using (var reader = new StreamReader(ms, Encoding.UTF8))
{
// do stuff
}
}
}
Cheers.

I might be wrong, but I don't think you're always supposed to receive a complete WebSocket message at once. The server may be sending the message in chunks (that'd correspond to calling SendAsync with endOfMessage: false).
So, do await _socket.ReceiveAsync() in a loop and accumulate the received chunks, until WebSocketReceiveResult.EndOfMessage is true or an error has occured.
On a side note, you probably should be using WebSocket.CreateClientBuffer instead of new ArraySegment<byte>(buffer).

// Read the bytes from the web socket and accumulate all into a list.
var buffer = new ArraySegment<byte>(new byte[1024]);
WebSocketReceiveResult result = null;
var allBytes = new List<byte>();
do
{
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
for (int i = 0; i < result.Count; i++)
{
allBytes.Add(buffer.Array[i]);
}
}
while (!result.EndOfMessage);
// Optional step to convert to a string (UTF-8 encoding).
var text = Encoding.UTF8.GetString(allBytes.ToArray(), 0, allBytes.Count);

Following Noseratio's answer I have implemented a temporary buffer that will construct the data of the entire message.
var temporaryBuffer = new byte[BufferSize];
var buffer = new byte[BufferSize * 20];
int offset = 0;
WebSocketReceiveResult response;
while (true)
{
response = await _socket.ReceiveAsync(
new ArraySegment<byte>(temporaryBuffer),
CancellationToken.None);
temporaryBuffer.CopyTo(buffer, offset);
offset += response.Count;
temporaryBuffer = new byte[BufferSize];
if (response.EndOfMessage)
{
break;
}
}
Full implementation here

Try this:
try
{
WebSocketReceiveResult result;
string receivedMessage = "";
var message = new ArraySegment<byte>(new byte[4096]);
do
{
result = await WebSocket.ReceiveAsync(message, DisconectToken);
if (result.MessageType != WebSocketMessageType.Text)
break;
var messageBytes = message.Skip(message.Offset).Take(result.Count).ToArray();
receivedMessage += Encoding.UTF8.GetString(messageBytes);
}
while (!result.EndOfMessage);
if (receivedMessage != "{}" && !string.IsNullOrEmpty(receivedMessage))
{
ResolveWebSocketResponse.Invoke(receivedMessage, Connection);
Console.WriteLine("Received: {0}", receivedMessage);
}
}
catch (Exception ex)
{
var mes = ex.Message;
}

How to implement send and receive hl7 data in .NET in ssh connection

I'm implementing an application in .Net. I have to create a connection by SSH which is works, but the HL7 data receiving fails. The destination is a raspberry pi. So when I'm debugging the ssh client is connected, the port is forwarded, the tcp client also connected, but there is no answer for my queries. Plese suggest me some examples!
In this project I have already implemented it on Android - it works fine.
So in .Net I tried the NHapiTools library and I also tried the direct TcpClient way too. localPort = remotePort. I used localIP = "localhost"
static void Main(string[] args)
{
try
{
PrivateKeyFile file = new PrivateKeyFile(#"./key/private.key");
using (var client = new SshClient(remoteIP, sshPort, username, file))
{
client.Connect();
var ci = client.ConnectionInfo;
var port = new ForwardedPortLocal(localIP, localPort, client.ConnectionInfo.Host, remotePort);
client.AddForwardedPort(port);
port.Start();
var req = "MSH|^~\\&|TestAppName||AVR||20181107201939.357+0000||QRY^R02^QRY_R02|923456|P|2.5";
////TCP
var tcpClient = new TcpClient();
tcpClient.Connect(localIP, (int)localPort);
Byte[] data = System.Text.Encoding.ASCII.GetBytes(req);
using (var stream = tcpClient.GetStream())
{
stream.Write(data, 0, data.Length);
using (var buffer = new MemoryStream())
{
byte[] chunk = new byte[4096];
int bytesRead;
while ((bytesRead = stream.Read(chunk, 0, chunk.Length)) > 0)
{
buffer.Write(chunk, 0, bytesRead);
}
data = buffer.ToArray();
}
}
//I used this also with same result -> no respond
//SimpleMLLP
/*
var connection = new SimpleMLLPClient(localIP, localPort,
Encoding.UTF8);
var response = connection.SendHL7Message(req);
*/
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
Console.ReadLine();
}
}
So I experinced that the buffer size is 0 in TCP (due to time out). In the SimpleMLLP test SendHK7Message method never returns

You are not implementing MLLP (also called LLP) protocol while sending message.
Description HEX ASCII Symbol
Message starting character 0B 11 <VT>
Message ending characters 1C,0D 28,13 <FS>,<CR>
This way, when you send a message to Listener (TCP/MLLP server), it looks for Start Block in your incoming data. It never finds it. It just discards your entire message considering garbage. Hence, you get nothing back from Listener.
With MLLP implemented, your message (the stuff you are writing on socket) should look something like below:
<VT>MSH|^~\\&|TestAppName||AVR||20181107201939.357+0000||QRY^R02^QRY_R02|923456|P|2.5<FS><CR>
Note the <VT>, <CR> and <FS> are place holders in above message.
You may refer to this article for detailed information (Read step 4 and onward):
using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
namespace SimpleMllpHl7ClientAdvanced
{
public class Program
{
private static char END_OF_BLOCK = '\u001c';
private static char START_OF_BLOCK = '\u000b';
private static char CARRIAGE_RETURN = (char)13;
static void Main(string[] args)
{
TcpClient ourTcpClient = null;
NetworkStream networkStream = null;
var testHl7MessageToTransmit = new StringBuilder();
//a HL7 test message that is enveloped with MLLP as described in my article
testHl7MessageToTransmit.Append(START_OF_BLOCK)
.Append("MSH|^~\\&|AcmeHIS|StJohn|CATH|StJohn|20061019172719||ORM^O01|MSGID12349876|P|2.3")
.Append(CARRIAGE_RETURN)
.Append("PID|||20301||Durden^Tyler^^^Mr.||19700312|M|||88 Punchward Dr.^^Los Angeles^CA^11221^USA|||||||")
.Append(CARRIAGE_RETURN)
.Append("PV1||O|OP^^||||4652^Paulson^Robert|||OP|||||||||9|||||||||||||||||||||||||20061019172717|20061019172718")
.Append(CARRIAGE_RETURN)
.Append("ORC|NW|20061019172719")
.Append(CARRIAGE_RETURN)
.Append("OBR|1|20061019172719||76770^Ultrasound: retroperitoneal^C4|||12349876")
.Append(CARRIAGE_RETURN)
.Append(END_OF_BLOCK)
.Append(CARRIAGE_RETURN);
try
{
//initiate a TCP client connection to local loopback address at port 1080
ourTcpClient = new TcpClient();
ourTcpClient.Connect(new IPEndPoint(IPAddress.Loopback, 1080));
Console.WriteLine("Connected to server....");
//get the IO stream on this connection to write to
networkStream = ourTcpClient.GetStream();
//use UTF-8 and either 8-bit encoding due to MLLP-related recommendations
var sendMessageByteBuffer = Encoding.UTF8.GetBytes(testHl7MessageToTransmit.ToString());
if (networkStream.CanWrite)
{
//send a message through this connection using the IO stream
networkStream.Write(sendMessageByteBuffer, 0, sendMessageByteBuffer.Length);
Console.WriteLine("Data was sent data to server successfully....");
var receiveMessageByteBuffer = Encoding.UTF8.GetBytes(testHl7MessageToTransmit.ToString());
var bytesReceivedFromServer = networkStream.Read(receiveMessageByteBuffer, 0, receiveMessageByteBuffer.Length);
// Our server for this example has been designed to echo back the message
// keep reading from this stream until the message is echoed back
while (bytesReceivedFromServer > 0)
{
if (networkStream.CanRead)
{
bytesReceivedFromServer = networkStream.Read(receiveMessageByteBuffer, 0, receiveMessageByteBuffer.Length);
if (bytesReceivedFromServer == 0)
{
break;
}
}
}
var receivedMessage = Encoding.UTF8.GetString(receiveMessageByteBuffer);
Console.WriteLine("Received message from server: {0}", receivedMessage);
}
Console.WriteLine("Press any key to exit...");
Console.ReadLine();
}
catch (Exception ex)
{
//display any exceptions that occur to console
Console.WriteLine(ex.Message);
}
finally
{
//close the IO strem and the TCP connection
networkStream?.Close();
ourTcpClient?.Close();
}
}
}
}
You should modify your following line of code as below:
var req = START_OF_BLOCK + "MSH|^~\\&|TestAppName||AVR||20181107201939.357+0000||QRY^R02^QRY_R02|923456|P|2.5" + END_OF_BLOCK + CARRIAGE_RETURN;
For more open source code, you may refer to this github project.

After days of struggling I have solved the problem. The main error was with the port forwarding.
I would reccomend to use SSH.Net by Renci (There was algorithm error with Tamir ssh).
After ssh connection created I used this to port forward:
var port = new ForwardedPortLocal(localIP, localPort, "localhost", remotePort);
Check your localIP with ipconfig /all in cmd. Or use 127.0.0.1 as a loopback IP.
SimpleMLLPClient did not worked for me so I used the direct tcp client query way. Like this:
TcpClient ourTcpClient = new TcpClient();
ourTcpClient.Connect(localIP, (int)localPort);
NetworkStream networkStream = ourTcpClient.GetStream();
var sendMessageByteBuffer = Encoding.UTF8.GetBytes(testHl7MessageToTransmit.ToString());
if (networkStream.CanWrite)
{
networkStream.Write(sendMessageByteBuffer, 0, sendMessageByteBuffer.Length);
Console.WriteLine("Data was sent to server successfully....");
byte[] receiveMessageByteBuffer = new byte[ourTcpClient.ReceiveBufferSize];
var bytesReceivedFromServer = networkStream.Read(receiveMessageByteBuffer, 0, receiveMessageByteBuffer.Length);
if (bytesReceivedFromServer > 0 && networkStream.CanRead)
{
receivedMessage.Append(Encoding.UTF8.GetString(receiveMessageByteBuffer));
}
var message = receivedMessage.Replace("\0", string.Empty);
Console.WriteLine("Received message from server: {0}", message);
}
So it gave me instant answer with 0 bytes (not due timeout). And here comes Amit Joshi help. I used a query what he suggested with START_OF_BLOCK, CARRIAGE_RETURN and END_OF_BLOCK and finally started to work. Thank you Amit Joshi!
Additional info:
In Android (java/Kotlin) jsch session setPortForwardingL works fine with three params:
val session = jsch.getSession("user", sshIP, sshPort)
session.setPassword("")
jsch.addIdentity(privatekey.getAbsolutePath())
// Avoid asking for key confirmation
val prop = Properties()
prop.setProperty("StrictHostKeyChecking", "no")
session.setConfig(prop)
session.connect(5000)
session.setPortForwardingL(localForwardPort, "localhost", remotePort)
val useTls = false
val context = DefaultHapiContext()
connection = context.newClient("localhost", localForwardPort, useTls)

NetworkStream in C# Socket programming

I am developing a IOS app in xamrain to communicate with hardware in which application sending the string message to the Hardware and hardware respond the message client reading that message through network-stream. After reading the message from the network-stream client flush the network-stream but old message still remain in network-stream. Plz suggest me the solution if anyone known.
private async Task CommandSendAsync()
{
if (sock.Connected)
{
StreamWriter writer = new StreamWriter(sock.GetStream());
writer.WriteLine("abc");
writer.Flush();
await DataReceivedAsync();
}
else { sock.Connect(192.168.1.1, 8025); }
}
string String = "";
public byte[] ReadJunctionSetting = new byte[1071];
int a = 0;
protected Socket SimNetSocket;
public async Task DataReceivedAsync()
{
try
{
await Task.Delay(2000);
String = "";
var abc = sock.GetStream();
//byte[] bytes = new byte[sock.ReceiveBufferSize];
while ((i = abc.Read(ReadJunctionSetting, 0, ReadJunctionSetting.Length)) != 0)
{
abc.Flush();
for (int j = 0; j < 1071; j++)
{
string c = char.ConvertFromUtf32(ReadJunctionSetting[j]);
String += c;
}
abc.Read(ReadJunctionSetting, 0, ReadJunctionSetting.Length);
break;
}
}
catch
{
}
}

From the .NET docs
The Flush method implements the Stream.Flush method; however, because NetworkStream is not buffered, it has no affect on network streams.
You can also the the released .NET source code for NetworkStream here.
Flush is an empty method.

I see several issues on your code, like #Damien_The_Unbeliever already pointed out. For example please avoid things like this:
string String = "";
Use a more meaningful variable name, like message.
But I think your problem is that you get the stream several times. You get then a new stream that still contains the already handled data.
Try to get the stream once after the connection is established and use than this instance for sending and receiving.

C# good practice waiting for TCP response

While making a c# application for remote controlling cisco routers using TCP, I got the problem of waiting for a response from the router.
For the application I have to connect to a Cisco router using a TCP connection. After the connection has been made a networkstream will push my command to the Cisco router. To let the router process the command, I am using Thread.Sleep. This is not the best solution.
Here is the complete code to get a idea what my program is doing.
string IPAddress = "192.168.1.1";
string message = "show running-config"; // command to run
int bytes;
string response = "";
byte[] responseInBytes = new byte[4096];
var client = new TcpClient();
client.ConnectAsync(IPAddress, 23).Wait(TimeSpan.FromSeconds(2));
if (client.Connected == true)
{
client.ReceiveTimeout = 3;
client.SendTimeout = 3;
byte[] messageInBytes = Encoding.ASCII.GetBytes(message);
NetworkStream stream = client.GetStream();
Console.WriteLine();
stream.Write(messageInBytes, 0, messageInBytes.Count()); //send data to router
Thread.Sleep(50); // temporary way to let the router fill his tcp response
bytes = stream.Read(responseInBytes, 0, responseInBytes.Length);
response = Encoding.ASCII.GetString(responseInBytes, 0, bytes);
return response; //whole command output
}
return null;
What is a good and reliable way to get the full response.
Thanks for any help or command.
More info:
The networksteam is always filled with something, most of the time it is filled with the cisco IOS login page. The biggest problem is to determine when the router is done filling up the response.
The response I most of the time get:
"??\u0001??\u0003??\u0018??\u001f\r\n\r\nUser Access Verification\r\n\r\nUsername: "
The return data will be diffent every time because it will be a result of a cisco command. This can vary from a short string to a very long string.
mrmathijs95 -

When reading from NetworkStream with Stream.Read it not 100% sure that you will read all expected data. Stream.Read can return when only few packet arrived and not waiting for others.
To be sure that you get all data use BinaryReader for reading.
BinaryReader.Read method will block current thread until all expected data arrived
private string GetResponse(string message)
{
const int RESPONSE_LENGTH = 4096;
byte[] messageInBytes = Encoding.ASCII.GetBytes(message);
bool leaveStreamOpen = true;
using(var writer = new BinaryWriter(client.GetStream()))
{
writer.Write(messageInBytes);
}
using(var reader = New BinaryReader(client.GetStream()))
{
byte[] bytes = reader.Read(RESPONSE_LENGTH );
return Encoding.ASCII.GetString(bytes);
}
}

Don't use Thread.Sleep. I would async/await the entire thing, given that you don't always know what the data is based on your recent edit. This is how I would do it (untested):
public class Program
{
// call Foo write with "show running-config"
}
public class Foo
{
private TcpClient _client;
private ConcurrentQueue<string> _responses;
private Task _continualRead;
private CancellationTokenSource _readCancellation;
public Foo()
{
this._responses = new ConcurrentQueue<string>();
this._readCancellation = new CancellationTokenSource();
this._continualRead = Task.Factory.StartNew(this.ContinualReadOperation, this._readCancellation.Token, this._readCancellation.Token);
}
public async Task<bool> Connect(string ip)
{
this._client = new TcpClient
{
ReceiveTimeout = 3, // probably shouldn't be 3ms.
SendTimeout = 3 // ^
};
int timeout = 1000;
return await this.AwaitTimeoutTask(this._client.ConnectAsync(ip, 23), timeout);
}
public async void StreamWrite(string message)
{
var messageBytes = Encoding.ASCII.GetBytes(message);
var stream = this._client.GetStream();
if (await this.AwaitTimeoutTask(stream.WriteAsync(messageBytes, 0, messageBytes.Length), 1000))
{
//write success
}
else
{
//write failure.
}
}
public async void ContinualReadOperation(object state)
{
var token = (CancellationToken)state;
var stream = this._client.GetStream();
var byteBuffer = new byte[4096];
while (!token.IsCancellationRequested)
{
int bytesLastRead = 0;
if (stream.DataAvailable)
{
bytesLastRead = await stream.ReadAsync(byteBuffer, 0, byteBuffer.Length, token);
}
if (bytesLastRead > 0)
{
var response = Encoding.ASCII.GetString(byteBuffer, 0, bytesLastRead);
this._responses.Enqueue(response);
}
}
}
private async Task<bool> AwaitTimeoutTask(Task task, int timeout)
{
return await Task.WhenAny(task, Task.Delay(timeout)) == task;
}
public void GetResponses()
{
//Do a TryDequeue etc... on this._responses.
}
}
I didn't expose the read cancellation publicly, but you could add this method to cancel the read operation:
public void Cancel()
{
this._readCancellation.Cancel();
}
And then dispose of your client and all that fun stuff.
Lastly, because you said there's always data available on the stream, where you're doing the read you may have to do some logic on the number of bytes last read to offset yourself within the stream if the data doesn't clear. You'll know if the responses you're getting is always the same.

This is the working code for me.
It uses the solution of Fabio,
combined with a while loop to check every X miliseconds if the response has changed.
client.ReceiveTimeout = 3;
client.SendTimeout = 3;
byte[] messageInBytes = Encoding.ASCII.GetBytes(message);
NetworkStream stream = client.GetStream();
Console.WriteLine();
using (var writer = new BinaryWriter(client.GetStream(),Encoding.ASCII,true))
{
writer.Write(messageInBytes);
}
using (var reader = new BinaryReader(client.GetStream(),Encoding.ASCII, true))
{
while (itIsTheEnd == false)
{
bytes = reader.Read(responseInBytes, 0, responseInBytes.Count());
if (lastBytesArray == responseInBytes)
{
itIsTheEnd = true;
}
lastBytesArray = responseInBytes;
Thread.Sleep(15);
}
}
response = Encoding.ASCII.GetString(responseInBytes);
Thanks for everyone who suggested a solution.
And thanks to Fabio for the given solution.

Read continous bytestream from Stream using TcpClient and Reactive Extensions

Consider the following code:
internal class Program
{
private static void Main(string[] args)
{
var client = new TcpClient();
client.ConnectAsync("localhost", 7105).Wait();
var stream = client.GetStream();
var observable = stream.ReadDataObservable().Repeat();
var s = from d in observable.Buffer(4)
let headerLength = IPAddress.NetworkToHostOrder(BitConverter.ToInt16(d.ToArray(), 2))
let b = observable.Take(headerLength)
select b.ToEnumerable().ToArray();
s.Subscribe(a => Console.WriteLine("{0}", a));
Console.ReadLine();
}
}
public static class Extensions
{
public static IObservable<byte> ReadDataObservable(this Stream stream)
{
return Observable.Defer(async () =>
{
var buffer = new byte[1024];
var readBytes = await stream.ReadAsync(buffer, 0, buffer.Length);
return buffer.Take(readBytes).ToObservable();
});
}
}
Basically I want to parse the messages I receive with Reactive Extensions. The header of the message is parsed correctly using the Buffer(4) and I get the length of the remainder of the message. The problem that arises is that when I do stream.Take(headerLength), the code reevaluates the whole "chain" and tries to get a new message from the stream instead of returning the rest of the bytes which already has been read from the stream. To be more exact, the first ReadAsync(...) returns 38 bytes, the Buffer(4) returns the first 4 of those, the observable.Take(headerLength) does not return the remainding 34 bytes but instead tries to read a new message with ReadAsync.
The question is, how can I make sure the observable.Take(headerLength) receives the already read 34 bytes and not try to read a new message from the stream? I've searched around for a solution, but I can't really figure out how to achieve this.
Edit: This solution (Using Reactive Extensions (Rx) for socket programming practical?) is not what I'm looking for. This isn't reading everything available in the stream (up to buffersize) and makes a continous bytestream out of it. To me this solution doesn't seem like a very efficient way to read from a stream, hence my question.

This approach isn't going to work. The problem is the way you are using the observable. Buffer will not read 4 bytes and quit, it will continually read 4 byte chunks. The Take forms a second subscription that will read overlapping bytes. You'll find it much easier to parse the stream directly into messages.
The following code makes a good deal of effort to clean up properly as well.
Assuming your Message is just this, (ToString added for testing):
public class Message
{
public byte[] PayLoad;
public override string ToString()
{
return Encoding.UTF8.GetString(PayLoad);
}
}
And you have acquired a Stream then you can parse it as follows. First, a method to read an exact number of bytes from a stream:
public async static Task ReadExactBytesAsync(
Stream stream, byte[] buffer, CancellationToken ct)
{
var count = buffer.Length;
var totalBytesRemaining = count;
var totalBytesRead = 0;
while (totalBytesRemaining != 0)
{
var bytesRead = await stream.ReadAsync(
buffer, totalBytesRead, totalBytesRemaining, ct);
ct.ThrowIfCancellationRequested();
totalBytesRead += bytesRead;
totalBytesRemaining -= bytesRead;
}
}
Then the conversion of a stream to IObservable<Message>:
public static IObservable<Message> ReadMessages(
Stream sourceStream,
IScheduler scheduler = null)
{
int subscribed = 0;
scheduler = scheduler ?? Scheduler.Default;
return Observable.Create<Message>(o =>
{
// first check there is only one subscriber
// (multiple stream readers would cause havoc)
int previous = Interlocked.CompareExchange(ref subscribed, 1, 0);
if (previous != 0)
o.OnError(new Exception(
"Only one subscriber is allowed for each stream."));
// we will return a disposable that cleans
// up both the scheduled task below and
// the source stream
var dispose = new CompositeDisposable
{
Disposable.Create(sourceStream.Dispose)
};
// use async scheduling to get nice imperative code
var schedule = scheduler.ScheduleAsync(async (ctrl, ct) =>
{
// store the header here each time
var header = new byte[4];
// loop until cancellation requested
while (!ct.IsCancellationRequested)
{
try
{
// read the exact number of bytes for a header
await ReadExactBytesAsync(sourceStream, header, ct);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
// pass through any problem in the stream and quit
o.OnError(new InvalidDataException("Error in stream.", ex));
return;
}
ct.ThrowIfCancellationRequested();
var bodyLength = IPAddress.NetworkToHostOrder(
BitConverter.ToInt16(header, 2));
// create buffer to read the message
var payload = new byte[bodyLength];
// read exact bytes as before
try
{
await ReadExactBytesAsync(sourceStream, payload, ct);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
o.OnError(new InvalidDataException("Error in stream.", ex));
return;
}
// create a new message and send it to client
var message = new Message { PayLoad = payload };
o.OnNext(message);
}
// wrap things up
ct.ThrowIfCancellationRequested();
o.OnCompleted();
});
// return the suscription handle
dispose.Add(schedule);
return dispose;
});
}
EDIT - Very hacky test code I used:
private static void Main(string[] args)
{
var listener = new TcpListener(IPAddress.Any, 12873);
listener.Start();
var listenTask = listener.AcceptTcpClientAsync();
listenTask.ContinueWith((Task<TcpClient> t) =>
{
var client = t.Result;
var stream = client.GetStream();
const string messageText = "Hello World!";
var body = Encoding.UTF8.GetBytes(messageText);
var header = BitConverter.GetBytes(
IPAddress.HostToNetworkOrder(body.Length));
for (int i = 0; i < 5; i++)
{
stream.Write(header, 0, 4);
stream.Write(body, 0, 4);
stream.Flush();
// deliberate nasty delay
Thread.Sleep(2000);
stream.Write(body, 4, body.Length - 4);
stream.Flush();
}
stream.Close();
listener.Stop();
});
var tcpClient = new TcpClient();
tcpClient.Connect(new IPEndPoint(IPAddress.Loopback, 12873));
var clientStream = tcpClient.GetStream();
ReadMessages(clientStream).Subscribe(
Console.WriteLine,
ex => Console.WriteLine("Error: " + ex.Message),
() => Console.WriteLine("Done!"));
Console.ReadLine();
}
Wrapping up
You need to think about setting a timeout for reads, in case the server dies, and some kind of "end message" should be sent by the server. Currently this method will just continually tries to receive bytes. As you haven't specced it, I haven't included anything like this - but if you do, then as I've written it just breaking out of the while loop will cause OnCompleted to be sent.

I guess what is needed here is Qactive: A Rx.Net based queryable reactive tcp server provider
Server
Observable
.Interval(TimeSpan.FromSeconds(1))
.ServeQbservableTcp(new IPEndPoint(IPAddress.Loopback, 3205))
.Subscribe();
Client
var datasourceAddress = new IPEndPoint(IPAddress.Loopback, 3205);
var datasource = new TcpQbservableClient<long>(datasourceAddress);
(
from value in datasource.Query()
//The code below is actually executed on the server
where value <= 5 || value >= 8
select value
)
.Subscribe(Console.WriteLine);
What´s mind blowing about this is that clients can say what and how frequently they want the data they receive and the server can still limit and control when, how frequent and how much data it returns.
For more info on this https://github.com/RxDave/Qactive
Another blog.sample
https://sachabarbs.wordpress.com/2016/12/23/rx-over-the-wire/