Consider the following code:
internal class Program
{
private static void Main(string[] args)
{
var client = new TcpClient();
client.ConnectAsync("localhost", 7105).Wait();
var stream = client.GetStream();
var observable = stream.ReadDataObservable().Repeat();
var s = from d in observable.Buffer(4)
let headerLength = IPAddress.NetworkToHostOrder(BitConverter.ToInt16(d.ToArray(), 2))
let b = observable.Take(headerLength)
select b.ToEnumerable().ToArray();
s.Subscribe(a => Console.WriteLine("{0}", a));
Console.ReadLine();
}
}
public static class Extensions
{
public static IObservable<byte> ReadDataObservable(this Stream stream)
{
return Observable.Defer(async () =>
{
var buffer = new byte[1024];
var readBytes = await stream.ReadAsync(buffer, 0, buffer.Length);
return buffer.Take(readBytes).ToObservable();
});
}
}
Basically I want to parse the messages I receive with Reactive Extensions. The header of the message is parsed correctly using the Buffer(4) and I get the length of the remainder of the message. The problem that arises is that when I do stream.Take(headerLength), the code reevaluates the whole "chain" and tries to get a new message from the stream instead of returning the rest of the bytes which already has been read from the stream. To be more exact, the first ReadAsync(...) returns 38 bytes, the Buffer(4) returns the first 4 of those, the observable.Take(headerLength) does not return the remainding 34 bytes but instead tries to read a new message with ReadAsync.
The question is, how can I make sure the observable.Take(headerLength) receives the already read 34 bytes and not try to read a new message from the stream? I've searched around for a solution, but I can't really figure out how to achieve this.
Edit: This solution (Using Reactive Extensions (Rx) for socket programming practical?) is not what I'm looking for. This isn't reading everything available in the stream (up to buffersize) and makes a continous bytestream out of it. To me this solution doesn't seem like a very efficient way to read from a stream, hence my question.
This approach isn't going to work. The problem is the way you are using the observable. Buffer will not read 4 bytes and quit, it will continually read 4 byte chunks. The Take forms a second subscription that will read overlapping bytes. You'll find it much easier to parse the stream directly into messages.
The following code makes a good deal of effort to clean up properly as well.
Assuming your Message is just this, (ToString added for testing):
public class Message
{
public byte[] PayLoad;
public override string ToString()
{
return Encoding.UTF8.GetString(PayLoad);
}
}
And you have acquired a Stream then you can parse it as follows. First, a method to read an exact number of bytes from a stream:
public async static Task ReadExactBytesAsync(
Stream stream, byte[] buffer, CancellationToken ct)
{
var count = buffer.Length;
var totalBytesRemaining = count;
var totalBytesRead = 0;
while (totalBytesRemaining != 0)
{
var bytesRead = await stream.ReadAsync(
buffer, totalBytesRead, totalBytesRemaining, ct);
ct.ThrowIfCancellationRequested();
totalBytesRead += bytesRead;
totalBytesRemaining -= bytesRead;
}
}
Then the conversion of a stream to IObservable<Message>:
public static IObservable<Message> ReadMessages(
Stream sourceStream,
IScheduler scheduler = null)
{
int subscribed = 0;
scheduler = scheduler ?? Scheduler.Default;
return Observable.Create<Message>(o =>
{
// first check there is only one subscriber
// (multiple stream readers would cause havoc)
int previous = Interlocked.CompareExchange(ref subscribed, 1, 0);
if (previous != 0)
o.OnError(new Exception(
"Only one subscriber is allowed for each stream."));
// we will return a disposable that cleans
// up both the scheduled task below and
// the source stream
var dispose = new CompositeDisposable
{
Disposable.Create(sourceStream.Dispose)
};
// use async scheduling to get nice imperative code
var schedule = scheduler.ScheduleAsync(async (ctrl, ct) =>
{
// store the header here each time
var header = new byte[4];
// loop until cancellation requested
while (!ct.IsCancellationRequested)
{
try
{
// read the exact number of bytes for a header
await ReadExactBytesAsync(sourceStream, header, ct);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
// pass through any problem in the stream and quit
o.OnError(new InvalidDataException("Error in stream.", ex));
return;
}
ct.ThrowIfCancellationRequested();
var bodyLength = IPAddress.NetworkToHostOrder(
BitConverter.ToInt16(header, 2));
// create buffer to read the message
var payload = new byte[bodyLength];
// read exact bytes as before
try
{
await ReadExactBytesAsync(sourceStream, payload, ct);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
o.OnError(new InvalidDataException("Error in stream.", ex));
return;
}
// create a new message and send it to client
var message = new Message { PayLoad = payload };
o.OnNext(message);
}
// wrap things up
ct.ThrowIfCancellationRequested();
o.OnCompleted();
});
// return the suscription handle
dispose.Add(schedule);
return dispose;
});
}
EDIT - Very hacky test code I used:
private static void Main(string[] args)
{
var listener = new TcpListener(IPAddress.Any, 12873);
listener.Start();
var listenTask = listener.AcceptTcpClientAsync();
listenTask.ContinueWith((Task<TcpClient> t) =>
{
var client = t.Result;
var stream = client.GetStream();
const string messageText = "Hello World!";
var body = Encoding.UTF8.GetBytes(messageText);
var header = BitConverter.GetBytes(
IPAddress.HostToNetworkOrder(body.Length));
for (int i = 0; i < 5; i++)
{
stream.Write(header, 0, 4);
stream.Write(body, 0, 4);
stream.Flush();
// deliberate nasty delay
Thread.Sleep(2000);
stream.Write(body, 4, body.Length - 4);
stream.Flush();
}
stream.Close();
listener.Stop();
});
var tcpClient = new TcpClient();
tcpClient.Connect(new IPEndPoint(IPAddress.Loopback, 12873));
var clientStream = tcpClient.GetStream();
ReadMessages(clientStream).Subscribe(
Console.WriteLine,
ex => Console.WriteLine("Error: " + ex.Message),
() => Console.WriteLine("Done!"));
Console.ReadLine();
}
Wrapping up
You need to think about setting a timeout for reads, in case the server dies, and some kind of "end message" should be sent by the server. Currently this method will just continually tries to receive bytes. As you haven't specced it, I haven't included anything like this - but if you do, then as I've written it just breaking out of the while loop will cause OnCompleted to be sent.
I guess what is needed here is Qactive: A Rx.Net based queryable reactive tcp server provider
Server
Observable
.Interval(TimeSpan.FromSeconds(1))
.ServeQbservableTcp(new IPEndPoint(IPAddress.Loopback, 3205))
.Subscribe();
Client
var datasourceAddress = new IPEndPoint(IPAddress.Loopback, 3205);
var datasource = new TcpQbservableClient<long>(datasourceAddress);
(
from value in datasource.Query()
//The code below is actually executed on the server
where value <= 5 || value >= 8
select value
)
.Subscribe(Console.WriteLine);
What´s mind blowing about this is that clients can say what and how frequently they want the data they receive and the server can still limit and control when, how frequent and how much data it returns.
For more info on this https://github.com/RxDave/Qactive
Another blog.sample
https://sachabarbs.wordpress.com/2016/12/23/rx-over-the-wire/
Related
I am receiving JSON through a websocket. At least: I am partially. Using an online websocket service I receive the full JSON response (all the HTML markup is ignored). When I look at the JSON that I receive in my console I can see the HTML markup (viewing it with the HTML viewer during debugging removes the HTML) but it ends abruptly (incomplete data).
My buffer has plenty of space and I am using async-await to (supposedly) wait for the entire response to come in before continuing.
private async Task Receive()
{
var buffer = new byte[4096 * 20];
while (_socket.State == WebSocketState.Open)
{
var response = await _socket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
if (response.MessageType == WebSocketMessageType.Close)
{
await
_socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Close response received",
CancellationToken.None);
}
else
{
var result = Encoding.UTF8.GetString(buffer);
var a = buffer[1000];
var b = buffer[10000];
var c = buffer[50000];
var d = buffer[81000];
Console.WriteLine(result);
var responseObject = JsonConvert.DeserializeObject<Response>(result, _requestParameters.ResponseDataType);
OnSocketReceive.Invoke(this, new SocketEventArgs {Response = responseObject });
buffer = new byte[4096 * 20];
}
}
}
Things to note: The buffer is perfectly big enough and b, c and d are never filled. I should also note that this only happens for the 1-questions-newest-tag-java request, 155-questions-active works perfectly fine.
After doing some digging I have found that response.CloseStatus and response.CloseStatusDescription are always null, response.Count is always 1396 (copy-pasting the result in Word does show that there are always 1396 characters) and response.EndOfMessage is false.
Digging through some source code I have found that the DefaultReceiveBufferSize is 16 * 1024 (big enough) and the WebSocketGetDefaultKeepAliveInterval() refers to an external implementation (but the debugger shows 00:00:30).
It is not a matter of timeout since the debugger halts at the same moment the online service receives its response.
Why is my method continuing to execute when the socket has not yet received all data?
Just to complete #Noseratio response, the code would be something like this:
ArraySegment<Byte> buffer = new ArraySegment<byte>(new Byte[8192]);
WebSocketReceiveResult result= null;
using (var ms = new MemoryStream())
{
do
{
result = await socket.ReceiveAsync(buffer, CancellationToken.None);
ms.Write(buffer.Array, buffer.Offset, result.Count);
}
while (!result.EndOfMessage);
ms.Seek(0, SeekOrigin.Begin);
if (result.MessageType == WebSocketMessageType.Text)
{
using (var reader = new StreamReader(ms, Encoding.UTF8))
{
// do stuff
}
}
}
Cheers.
I might be wrong, but I don't think you're always supposed to receive a complete WebSocket message at once. The server may be sending the message in chunks (that'd correspond to calling SendAsync with endOfMessage: false).
So, do await _socket.ReceiveAsync() in a loop and accumulate the received chunks, until WebSocketReceiveResult.EndOfMessage is true or an error has occured.
On a side note, you probably should be using WebSocket.CreateClientBuffer instead of new ArraySegment<byte>(buffer).
// Read the bytes from the web socket and accumulate all into a list.
var buffer = new ArraySegment<byte>(new byte[1024]);
WebSocketReceiveResult result = null;
var allBytes = new List<byte>();
do
{
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
for (int i = 0; i < result.Count; i++)
{
allBytes.Add(buffer.Array[i]);
}
}
while (!result.EndOfMessage);
// Optional step to convert to a string (UTF-8 encoding).
var text = Encoding.UTF8.GetString(allBytes.ToArray(), 0, allBytes.Count);
Following Noseratio's answer I have implemented a temporary buffer that will construct the data of the entire message.
var temporaryBuffer = new byte[BufferSize];
var buffer = new byte[BufferSize * 20];
int offset = 0;
WebSocketReceiveResult response;
while (true)
{
response = await _socket.ReceiveAsync(
new ArraySegment<byte>(temporaryBuffer),
CancellationToken.None);
temporaryBuffer.CopyTo(buffer, offset);
offset += response.Count;
temporaryBuffer = new byte[BufferSize];
if (response.EndOfMessage)
{
break;
}
}
Full implementation here
Try this:
try
{
WebSocketReceiveResult result;
string receivedMessage = "";
var message = new ArraySegment<byte>(new byte[4096]);
do
{
result = await WebSocket.ReceiveAsync(message, DisconectToken);
if (result.MessageType != WebSocketMessageType.Text)
break;
var messageBytes = message.Skip(message.Offset).Take(result.Count).ToArray();
receivedMessage += Encoding.UTF8.GetString(messageBytes);
}
while (!result.EndOfMessage);
if (receivedMessage != "{}" && !string.IsNullOrEmpty(receivedMessage))
{
ResolveWebSocketResponse.Invoke(receivedMessage, Connection);
Console.WriteLine("Received: {0}", receivedMessage);
}
}
catch (Exception ex)
{
var mes = ex.Message;
}
I am reading data in bytes from .bin file and split the whole byte data into 16-16 bytes frames, so I want to 16 bytes frame one by one and wait until the first frame finished its cycle.
Callback method of SerialPort class:
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
// Read data from serial port:
byte[] buffer = new byte[serialPort.BytesToRead];
serialPort.Read(buffer, 0, buffer.Length);
StringBuilder sb = new StringBuilder();
List<string> response = new List<string>();
for (int i = 0; i < buffer.Length; i++)
{
string currentByte = string.Format("{0:X2}", buffer[i]);
response.Add(currentByte);
sb.AppendFormat("{0:X2}", buffer[i]);
}
string responesCode = response[1].ToString();
if (responesCode == "44")
{
// Wait until the first response is not received
foreach (var packet in packetList.Skip(1))
{
// This method which sending the the data
this.ReadByteDataFromFile(packet);
}
}
}
FdBrowseFile_Click button click:
private void FdBrowseFile_Click(object sender, RoutedEventArgs e)
{
Microsoft.Win32.OpenFileDialog dlg = new Microsoft.Win32.OpenFileDialog();
Nullable<bool> result = dlg.ShowDialog();
if (result == true)
{
byte[] fileBytes = File.ReadAllBytes(filename);
foreach (byte[] copySlice in fileBytes.Slices(16))
{
var splitedByteArray = copySlice;
if (splitedByteArray.Length != 16)
{
byte[] padd = new byte[16];
var startAt = 0;
Array.Copy(splitedByteArray, 0, padd, startAt, splitedByteArray.Length);
packetList.Add(padd);
}
else
{
packetList.Add(splitedByteArray);
}
}
ReadByteDataFromFile(packetList[0]);
}
}
ReadByteDataFromFile method:
public void ReadByteDataFromFile(byte[] packet) {
try {
byte[] mBuffer = new byte[24];
byte[] payload = new byte[16];
int i = 0;
foreach(var bytes in packet) {
payload[i++] = bytes;
}
CheckSumHelper checkSumHelper = new CheckSumHelper();
var ckSum = checkSumHelper.GetCheckSum(payload);
mBuffer[0] = 0x02;
mBuffer[1] = 0x10;
mBuffer[2] = CheckSumHelper.GetBytesFromDecimal(packet[0]);
mBuffer[3] = CheckSumHelper.GetBytesFromDecimal(packet[1]);
mBuffer[4] = CheckSumHelper.GetBytesFromDecimal(packet[2]);
mBuffer[5] = CheckSumHelper.GetBytesFromDecimal(packet[3]);
mBuffer[6] = CheckSumHelper.GetBytesFromDecimal(packet[4]);
mBuffer[7] = CheckSumHelper.GetBytesFromDecimal(packet[5]);
mBuffer[8] = CheckSumHelper.GetBytesFromDecimal(packet[6]);
mBuffer[9] = CheckSumHelper.GetBytesFromDecimal(packet[7]);
mBuffer[10] = CheckSumHelper.GetBytesFromDecimal(packet[8]);
mBuffer[11] = CheckSumHelper.GetBytesFromDecimal(packet[9]);
mBuffer[12] = CheckSumHelper.GetBytesFromDecimal(packet[10]);
mBuffer[13] = CheckSumHelper.GetBytesFromDecimal(packet[11]);
mBuffer[14] = CheckSumHelper.GetBytesFromDecimal(packet[12]);
mBuffer[15] = CheckSumHelper.GetBytesFromDecimal(packet[13]);
mBuffer[16] = CheckSumHelper.GetBytesFromDecimal(packet[14]);
mBuffer[17] = CheckSumHelper.GetBytesFromDecimal(packet[15]);
mBuffer[18] = 0x17;
mBuffer[19] = 0x00;
mBuffer[20] = 0x00;
mBuffer[21] = 0x00;
mBuffer[22] = Convert.ToByte(int.Parse(ckSum, System.Globalization.NumberStyles.HexNumber));
mBuffer[23] = 0x03;
serialPort.Write(mBuffer, 0, mBuffer.Length);
} catch (Exception ex) {
ExceptionHandler exceptionHandler = new ExceptionHandler();
exceptionHandler.HandleException(ex);
}
}
How I can add a delay for ReadByteDataFromFile method?
What you need is a way to block the execution of some code, until something else has happened, or - how to get things running on two threads synchronously.
.NET has quite a few classes in the System.Threading namespace for synchronization. We'll use AutoResetEvent here.
Think of AutoResetEvent as a turnstile.
You can't move forward if the the person on the other side stops.
When you move forward, you call Wait - and it blocks you from moving,
until someone calls Set on it.
Now if we apply that to our problem:
We need to stop sending data until we get an acceptable response.
So call Wait when sending data, and let the response handling code call Set to let it move forward.
Here's an example which simulates a modem.
You send some AT commands, it responds, but the responses always end with \r\n.
var port = new SerialPort("COM2");
port.Open();
var mre = new AutoResetEvent(false);
var buffer = new StringBuilder();
port.DataReceived += (s, e) =>
{
buffer.Append(port.ReadExisting());
if (buffer.ToString().IndexOf("\r\n") >= 0)
{
Console.WriteLine("Got response: {0}", buffer);
mre.Set(); //allow loop to continue
buffer.Clear();
}
};
var commandsToSend = new string[] { "AT", "AT", "AT+CSQ" };
var responseTimeout = TimeSpan.FromSeconds(10);
foreach (var command in commandsToSend)
{
try
{
Console.WriteLine("Write '{0}' to {1}", command, port.PortName);
port.WriteLine(command);
Console.WriteLine("Waiting for response...");
//this is where we block
if (!mre.WaitOne(responseTimeout))
{
Console.WriteLine("Did not receive response");
//do something
}
}
catch (TimeoutException)
{
Console.WriteLine("Write took longer than expected");
}
catch
{
Console.WriteLine("Failed to write to port");
}
}
Console.ReadLine();
Sample output when tested through virtual serial port:
(I just reply with OK<CR><LF>)
Write 'AT' to COM2
Waiting for response...
Got response: OK
Write 'AT' to COM2
Waiting for response...
Got response: OK
Write 'AT+CSQ' to COM2
Waiting for response...
Did not receive response
Wait in a loop for a full response after you write a first frame.
// Set read timeout to value recommended in the communication protocol specification
// so serial port operations don't stuck.
_port.WriteTimeout = 200;
_port.ReadTimeout = 200;
public void OnClick()
{
// Write first frame.
_port.Write(...);
// Now wait for the full response.
// Expected response length. Look for the constant value from the device communication
// protocol specification or extract from the response header (first response bytes) if
// there is any specified in the protocol.
int count = ...;
var buffer = new byte[count];
var offset = 0;
while (count > 0)
{
var readCount = _port.Read(buffer, offset, count);
offset += readCount;
count -= readCount;
}
// Now buffer contains full response or TimeoutException instance is thrown by SerialPort.
// Check response status code and write other frames.
}
In order to not block UI thread you most probably still need to utilize synchronous API and Task.Run(). See C# await event and timeout in serial port communication discussion on StackOverflow.
For more information check Top 5 SerialPort Tips article by Kim Hamilton.
I have implemented a socket client using TcpClient class. So I can send and receive data, everything works good. But I ask some of the guru's out there :) Is there something wrong with my implementation? maybe there is a better way of doing things. In particular, how do I handle disconnects? Is there some indicator (or maybe I can write one myself) that tells me that socket has disconnected?
I also have looked into async await features of Socket class, but can't wrap my head around "SocketAsyncEventArgs", why is it there in the first place.
Why cant i just: await Client.SendAsync("data"); ?
public class Client
{
private TcpClient tcpClient;
public void Initialize(string ip, int port)
{
try
{
tcpClient = new TcpClient(ip, port);
if (tcpClient.Connected)
Console.WriteLine("Connected to: {0}:{1}", ip, port);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Initialize(ip, port);
}
}
public void BeginRead()
{
var buffer = new byte[4096];
var ns = tcpClient.GetStream();
ns.BeginRead(buffer, 0, buffer.Length, EndRead, buffer);
}
public void EndRead(IAsyncResult result)
{
var buffer = (byte[])result.AsyncState;
var ns = tcpClient.GetStream();
var bytesAvailable = ns.EndRead(result);
Console.WriteLine(Encoding.ASCII.GetString(buffer, 0, bytesAvailable));
BeginRead();
}
public void BeginSend(string xml)
{
var bytes = Encoding.ASCII.GetBytes(xml);
var ns = tcpClient.GetStream();
ns.BeginWrite(bytes, 0, bytes.Length, EndSend, bytes);
}
public void EndSend(IAsyncResult result)
{
var bytes = (byte[])result.AsyncState;
Console.WriteLine("Sent {0} bytes to server.", bytes.Length);
Console.WriteLine("Sent: {0}", Encoding.ASCII.GetString(bytes));
}
}
And the usage:
static void Main(string[] args)
{
var client = new Client();
client.Initialize("127.0.0.1", 8778);
client.BeginRead();
client.BeginSend("<Names><Name>John</Name></Names>");
Console.ReadLine();
}
Okay it took me 10 seconds to find biggest issue you could've done :
public void BeginRead()
{
var buffer = new byte[4096];
var ns = tcpClient.GetStream();
ns.BeginRead(buffer, 0, buffer.Length, EndRead, buffer);
}
But don't worry that's why we are on SO.
First let me explain why it's such a big issue.
Assume that you're sending message which is 4097 bytes long. Your buffer can only accept 4096 bytes meaning that you cannot pack whole message into this buffer.
Assume you're sending message which is 12 bytes long. You're still allocating 4096 bytes in your memory just to store 12 bytes.
How to deal with this?
Every time you work with networking you should consider making some kind of protocol ( some people call it message framing, but it's just a protocol ) that will help you to identify incomming package.
Example of a protocol could be :
[1B = type of message][4B = length][XB = message]
- where X == BitConvert.ToInt32(length);
Receiver:
byte messageType = (byte)netStream.ReadByte();
byte[] lengthBuffer = new byte[sizeof(int)];
int recv = netStream.Read(lengthBuffer, 0, lengthBuffer.Length);
if(recv == sizeof(int))
{
int messageLen = BitConverter.ToInt32(lengthBuffer, 0);
byte[] messageBuffer = new byte[messageLen];
recv = netStream.Read(messageBuffer, 0, messageBuffer.Length);
if(recv == messageLen)
{
// messageBuffer contains your whole message ...
}
}
Sender:
byte messageType = (1 << 3); // assume that 0000 1000 would be XML
byte[] message = Encoding.ASCII.GetBytes(xml);
byte[] length = BitConverter.GetBytes(message.Length);
byte[] buffer = new byte[sizeof(int) + message.Length + 1];
buffer[0] = messageType;
for(int i = 0; i < sizeof(int); i++)
{
buffer[i + 1] = length[i];
}
for(int i = 0; i < message.Length; i++)
{
buffer[i + 1 + sizeof(int)] = message[i];
}
netStream.Write(buffer);
Rest of your code looks okay. But in my opinion using asynchronous operations in your case is just useless. You can do the same with synchronous calls.
It's hard to answer because theres no exact question here but more some kind of code review. But still some hints:
Your connect mechanism seems wrong. I don't think TcpClient.Connected will block until the connection was established. So it will often just fail as the connection is in progress and then you start all over again. You should switch to using a blocking or async Connect method.
SocketAsyncEventArgs is a mechanism for high performance async data transmission. There's rarely a need for it. You should just ignore it
If you want to send data asynchronously you should use the Async methods which return a Task, because these can be easily combined with async/await.
The APM model (BeginXYZ/EndXYZ) is kind of deprecated, you shouldn't use it anymore in new code. One issue with it is that sometimes the End method is called synchronously inside the Begin method which can lead to surprising behavior. If this is not the case the completion callback will be performed from a random thread on the ThreadPool. This is also often not what you want. The TPL methods avoid this.
For your simple use case the blocking methods are also totally fine and do not come with the complexity of the various async methods.
The reading side of the code with TPL methods (untested):
public async Task Initialize(string ip, int port)
{
tcpClient = new TcpClient;
await tcpClient.ConnectAsync(ip, port);
Console.WriteLine("Connected to: {0}:{1}", ip, port);
}
public async Task Read()
{
var buffer = new byte[4096];
var ns = tcpClient.GetStream();
while (true)
{
var bytesRead = await ns.ReadAsync(buffer, 0, buffer.Length);
if (bytesRead == 0) return; // Stream was closed
Console.WriteLine(Encoding.ASCII.GetString(buffer, 0, bytesRead));
}
}
In the initialization part you would do:
await client.Initialize(ip, port);
// Start reading task
Task.Run(() => client.Read());
For using synchronous methods delete all Async occurences and replace Task with Thread.
While making a c# application for remote controlling cisco routers using TCP, I got the problem of waiting for a response from the router.
For the application I have to connect to a Cisco router using a TCP connection. After the connection has been made a networkstream will push my command to the Cisco router. To let the router process the command, I am using Thread.Sleep. This is not the best solution.
Here is the complete code to get a idea what my program is doing.
string IPAddress = "192.168.1.1";
string message = "show running-config"; // command to run
int bytes;
string response = "";
byte[] responseInBytes = new byte[4096];
var client = new TcpClient();
client.ConnectAsync(IPAddress, 23).Wait(TimeSpan.FromSeconds(2));
if (client.Connected == true)
{
client.ReceiveTimeout = 3;
client.SendTimeout = 3;
byte[] messageInBytes = Encoding.ASCII.GetBytes(message);
NetworkStream stream = client.GetStream();
Console.WriteLine();
stream.Write(messageInBytes, 0, messageInBytes.Count()); //send data to router
Thread.Sleep(50); // temporary way to let the router fill his tcp response
bytes = stream.Read(responseInBytes, 0, responseInBytes.Length);
response = Encoding.ASCII.GetString(responseInBytes, 0, bytes);
return response; //whole command output
}
return null;
What is a good and reliable way to get the full response.
Thanks for any help or command.
More info:
The networksteam is always filled with something, most of the time it is filled with the cisco IOS login page. The biggest problem is to determine when the router is done filling up the response.
The response I most of the time get:
"??\u0001??\u0003??\u0018??\u001f\r\n\r\nUser Access Verification\r\n\r\nUsername: "
The return data will be diffent every time because it will be a result of a cisco command. This can vary from a short string to a very long string.
mrmathijs95 -
When reading from NetworkStream with Stream.Read it not 100% sure that you will read all expected data. Stream.Read can return when only few packet arrived and not waiting for others.
To be sure that you get all data use BinaryReader for reading.
BinaryReader.Read method will block current thread until all expected data arrived
private string GetResponse(string message)
{
const int RESPONSE_LENGTH = 4096;
byte[] messageInBytes = Encoding.ASCII.GetBytes(message);
bool leaveStreamOpen = true;
using(var writer = new BinaryWriter(client.GetStream()))
{
writer.Write(messageInBytes);
}
using(var reader = New BinaryReader(client.GetStream()))
{
byte[] bytes = reader.Read(RESPONSE_LENGTH );
return Encoding.ASCII.GetString(bytes);
}
}
Don't use Thread.Sleep. I would async/await the entire thing, given that you don't always know what the data is based on your recent edit. This is how I would do it (untested):
public class Program
{
// call Foo write with "show running-config"
}
public class Foo
{
private TcpClient _client;
private ConcurrentQueue<string> _responses;
private Task _continualRead;
private CancellationTokenSource _readCancellation;
public Foo()
{
this._responses = new ConcurrentQueue<string>();
this._readCancellation = new CancellationTokenSource();
this._continualRead = Task.Factory.StartNew(this.ContinualReadOperation, this._readCancellation.Token, this._readCancellation.Token);
}
public async Task<bool> Connect(string ip)
{
this._client = new TcpClient
{
ReceiveTimeout = 3, // probably shouldn't be 3ms.
SendTimeout = 3 // ^
};
int timeout = 1000;
return await this.AwaitTimeoutTask(this._client.ConnectAsync(ip, 23), timeout);
}
public async void StreamWrite(string message)
{
var messageBytes = Encoding.ASCII.GetBytes(message);
var stream = this._client.GetStream();
if (await this.AwaitTimeoutTask(stream.WriteAsync(messageBytes, 0, messageBytes.Length), 1000))
{
//write success
}
else
{
//write failure.
}
}
public async void ContinualReadOperation(object state)
{
var token = (CancellationToken)state;
var stream = this._client.GetStream();
var byteBuffer = new byte[4096];
while (!token.IsCancellationRequested)
{
int bytesLastRead = 0;
if (stream.DataAvailable)
{
bytesLastRead = await stream.ReadAsync(byteBuffer, 0, byteBuffer.Length, token);
}
if (bytesLastRead > 0)
{
var response = Encoding.ASCII.GetString(byteBuffer, 0, bytesLastRead);
this._responses.Enqueue(response);
}
}
}
private async Task<bool> AwaitTimeoutTask(Task task, int timeout)
{
return await Task.WhenAny(task, Task.Delay(timeout)) == task;
}
public void GetResponses()
{
//Do a TryDequeue etc... on this._responses.
}
}
I didn't expose the read cancellation publicly, but you could add this method to cancel the read operation:
public void Cancel()
{
this._readCancellation.Cancel();
}
And then dispose of your client and all that fun stuff.
Lastly, because you said there's always data available on the stream, where you're doing the read you may have to do some logic on the number of bytes last read to offset yourself within the stream if the data doesn't clear. You'll know if the responses you're getting is always the same.
This is the working code for me.
It uses the solution of Fabio,
combined with a while loop to check every X miliseconds if the response has changed.
client.ReceiveTimeout = 3;
client.SendTimeout = 3;
byte[] messageInBytes = Encoding.ASCII.GetBytes(message);
NetworkStream stream = client.GetStream();
Console.WriteLine();
using (var writer = new BinaryWriter(client.GetStream(),Encoding.ASCII,true))
{
writer.Write(messageInBytes);
}
using (var reader = new BinaryReader(client.GetStream(),Encoding.ASCII, true))
{
while (itIsTheEnd == false)
{
bytes = reader.Read(responseInBytes, 0, responseInBytes.Count());
if (lastBytesArray == responseInBytes)
{
itIsTheEnd = true;
}
lastBytesArray = responseInBytes;
Thread.Sleep(15);
}
}
response = Encoding.ASCII.GetString(responseInBytes);
Thanks for everyone who suggested a solution.
And thanks to Fabio for the given solution.
I'm sending 10 messages from a client to a server in a loop, and then responses back from the server to the client using TPL's FromAsync (targeting .NET 4.0)
After the last message is sent from the client, I'm trying to measure the total time taken until the final response is received. To do this, I call Task.WaitAll but I get an AggregateException who's inner exception says "A task was cancelled" for all the tasks. The server is receiving the requests.
Code:
private static double MeasureAsyncTPL(List<string> stringList)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
var tasks = new Task[stringList.Count];
for (int i = 0; i < stringList.Count; i++)
{
string source = stringList[i];
tasks[i] = SendMessageToServerAsync(source);
//SendMessageToServerAsync(source);
}
Task.WaitAll(tasks);
stopwatch.Stop();
return stopwatch.Elapsed.TotalMilliseconds;
}
static Task SendMessageToServerAsync(string message)
{
Task task;
var byteArray = MessageToByteArray(message, Encoding.UTF8);
using (var tcpClient = new TcpClient())
{
tcpClient.Connect("127.0.0.1", 5000);
using (var networkStream = tcpClient.GetStream())
{
task = Write(networkStream,byteArray,0);
Task continuation = task.ContinueWith(ant => Read(ant, networkStream), TaskContinuationOptions.NotOnFaulted | TaskContinuationOptions.AttachedToParent);
return continuation;
}
}
}
private static Task Write(NetworkStream stream, byte[] buffer, int offset)
{
return Task.Factory.FromAsync(stream.BeginWrite, stream.EndWrite, buffer, offset, buffer.Length, null);
}
private static Task Read(Task write, NetworkStream stream)
{
byte[] data = new byte[50];
return Task<int>.Factory.FromAsync(stream.BeginRead, stream.EndRead, data, 0, data.Length, null);
}
The exception is being picked up inside MeasureAsyncTPL on the line Task.WaitAll(tasks). How can I fix this exception?
Second question - is FromAsync inside my Read method guaranteed to return the entire message from the server? Or do I need to call it multiple times while keeping track of a the size of the byte array that the server sent?
You are disposing of the networkStream and tcpClient before the write and read operations start or have time to work.
With .NET 4.0, you need to do something as complex as this (this is something to start with, it needs more work):
var byteArray = MessageToByteArray(message, Encoding.UTF8);
var tcpClient = new TcpClient();
NetworkStream networkStream = null;
try
{
tcpClient.Connect("127.0.0.1", 5000);
networkStream = tcpClient.GetStream();
}
catch
{
((IDisposable)tcpClient).Dispose();
throw;
}
byte[] read_buffer = new byte[50];
Action dispose_action = () =>
{
((IDisposable)tcpClient).Dispose();
networkStream.Dispose();
};
var write_task = networkStream.WriteAsync(byteArray, 0, byteArray.Length);
write_task
.ContinueWith(wt =>
{
var read_task = networkStream.ReadAsync(read_buffer, 0, read_buffer.Length);
read_task.ContinueWith(rt =>
{
//See if rt task is successful and if so, process read data here
dispose_action();
});
},
TaskContinuationOptions.OnlyOnRanToCompletion);
write_task.ContinueWith(wt => dispose_action, TaskContinuationOptions.NotOnRanToCompletion);
The main idea of the code above is that you don't dispose of the TCP client and the network stream object until the read/write operations are done.
Imagine how complex this code would become if you want to read multiple times in some kind of loop.
With .NET 4.5, you can use async and await to do something (simpler) like this:
static async Task SendMessageToServerAsync(string message)
{
var byteArray = MessageToByteArray(message, Encoding.UTF8);
using (var tcpClient = new TcpClient())
{
tcpClient.Connect("127.0.0.1", 5000);
using (var networkStream = tcpClient.GetStream())
{
await networkStream.WriteAsync(byteArray, 0, byteArray.Length);
byte[] read_buffer = new byte[50];
int read = await networkStream.ReadAsync(read_buffer, 0, read_buffer.Length);
// do something with the read_buffer
}
}
}