I'm trying to read strings from a TCP socket. The server (which I have no access to) sends strings that can be smaller (only 3 chars) or longer( > 10.000 chars) than the BUFFER_SIZE.
For that I'm using the following method, which runs continuously in a separate task (thread). As soon as it reads a valid string it triggers an event, which the main task is subscribed to [...]
The cTS (CancellationTokenSource) is required for external cancellation of the reader.ReadAsync(...)
private readonly int BUFFER_SIZE = 4096;
private StreamSocket tcpSocket;
private StreamReader reader;
private CancellationTokenSource cTS;
public string readMessageFromServer()
{
string result = "";
while(true)
{
char[] buffer = new char[BUFFER_SIZE];
cTS = new CancellationTokenSource();
reader.ReadAsync(buffer, 0, BUFFER_SIZE).Wait(cTS.Token);
string data = new string(buffer);
if (data.IndexOf("\0") >= 0 || reader.EndOfStream)
{
return result + data.Substring(0, data.IndexOf("\0"));
}
result += data;
}
}
My Code works just fine for strings with a length < 4096 chars, but as soon as the string gets longer than 4096 chars it fails (won't detect the message end). It waits until it receives a new string < 4096 chars, concatenates both strings and returns them as one string.
Is there a way to get the actual length of a string and read them successively?
reader.ReadLineAsync()
Will not work, because the strings can contain "\n" or "\r\n" and the server won't add line breaks at the end of each string
I finally figured out to read long messages:
I had to user DataReader and DataWriter instead of StreamReader and StreamWriter.
/// <summary>
/// How many characters should get read at once max.
/// </summary>
private static readonly int BUFFER_SIZE = 4096;
private StreamSocket socket;
private DataReader dataReader;
private DataWriter dataWriter;
public string readNextString() {
string result = "";
readingCTS = new CancellationTokenSource();
try {
uint readCount = 0;
// Read the first batch:
Task < uint > t = dataReader.LoadAsync(BUFFER_SIZE).AsTask();
t.Wait(readingCTS.Token);
readCount = t.Result;
if (dataReader == null) {
return result;
}
while (dataReader.UnconsumedBufferLength > 0) {
result +=dataReader.ReadString(dataReader.UnconsumedBufferLength);
}
// If there is still data left to read, continue until a timeout occurs or a close got requested:
while (!readingCTS.IsCancellationRequested && readCount >= BUFFER_SIZE) {
t = dataReader.LoadAsync(BUFFER_SIZE).AsTask();
t.Wait(100, readingCTS.Token);
readCount = t.Result;
while (dataReader.UnconsumedBufferLength > 0) {
result += dataReader.ReadString(dataReader.UnconsumedBufferLength);
}
}
}
catch(AggregateException) {}
catch(NullReferenceException) {}
return result;
}
Related
I'm trying to communicate with an XMPP (Jabber) server via a TCP network socket (StreamSocket) and I'm using the following code to read what the server has send to me:
StreamSocket tcpSocket;
StreamReader reader;
int BUFFER_SIZE = 4096;
// Connecting to a remote XMPP server ....
reader = new StreamReader(tcpSocket.InputStream.AsStreamForRead());
string result;
while (true)
{
result = "";
while (true)
{
char[] buffer = new char[BUFFER_SIZE];
await reader.ReadAsync(buffer, 0, BUFFER_SIZE);
string data = new string(buffer);
// Detecting if all elements in the buffer array got replaced => there is more to read
if (data.IndexOf("\0") >= 0 || reader.EndOfStream)
{
result + data.Substring(0, data.IndexOf("\0"));
break;
}
result += data;
}
Debug.WriteLine(result);
}
My Code works just fine for strings with a length < 4096 chars, but as soon as the string gets longer than 4096 chars it fails (won't detect the message end). It waits until it receives a new string < 4096 chars, concatenates both strings and returns them as one string.
Is there a way to get the actual length of a string and read them successively?
You have set 4096 to the BUFFER_SIZE and it is be set to the count parameter in StreamReader.ReadAsync and the char Array. When the string contain more than 4096 chars, it will fails.
You should be able to get the actual length in the Stream, we can use Stream.Length to get the length of the stream in bytes. The last char of Array is "\0". When you create the char Array, you should be able to set the Stream.Length plus one to the char Array.
For example:
StreamSocket socket;
StreamSocket tcpSocket;
StreamReader reader;
reader = new StreamReader(tcpSocket.InputStream.AsStreamForRead());
var BUFFER_SIZE=(int)(tcpSocket.InputStream.AsStreamForRead()).Length;
string result;
while (true)
{
result = "";
while (true)
{
char[] buffer = new char[BUFFER_SIZE+1];
await reader.ReadAsync(buffer, 0, BUFFER_SIZE);
string data = new string(buffer);
if (data.IndexOf("\0") >= 0 || reader.EndOfStream)
{
result = data.Substring(0, data.IndexOf("\0"));
break;
}
result += data;
}
Debug.WriteLine(result);
}
If you want to reads all characters from the current position to the end of the stream, you can use StreamReader.ReadToEnd or StreamReader.ReadToEndAsync method.
I finally figured out to read long messages:
I had to user DataReader and DataWriter instead of StreamReader and StreamWriter.
/// <summary>
/// How many characters should get read at once max.
/// </summary>
private static readonly int BUFFER_SIZE = 4096;
private StreamSocket socket;
private DataReader dataReader;
private DataWriter dataWriter;
public string readNextString() {
string result = "";
readingCTS = new CancellationTokenSource();
try {
uint readCount = 0;
// Read the first batch:
Task < uint > t = dataReader.LoadAsync(BUFFER_SIZE).AsTask();
t.Wait(readingCTS.Token);
readCount = t.Result;
if (dataReader == null) {
return result;
}
while (dataReader.UnconsumedBufferLength > 0) {
result +=dataReader.ReadString(dataReader.UnconsumedBufferLength);
}
// If there is still data left to read, continue until a timeout occurs or a close got requested:
while (!readingCTS.IsCancellationRequested && readCount >= BUFFER_SIZE) {
t = dataReader.LoadAsync(BUFFER_SIZE).AsTask();
t.Wait(100, readingCTS.Token);
readCount = t.Result;
while (dataReader.UnconsumedBufferLength > 0) {
result += dataReader.ReadString(dataReader.UnconsumedBufferLength);
}
}
}
catch(AggregateException) {}
catch(NullReferenceException) {}
return result;
}
i am facing problem in reading the full content from the response, the length of the decoder is not coming as per the length of the response content, i am having problem in line
int read = responseStream.EndRead(asyncResult);
//the value of read is very less in accordance to the response.
int len = rs.StreamDecode.GetChars(rs.BufferRead, 0, read, charBuffer, 0);
below is my code for the same
private void ReadQtoRCallBack(IAsyncResult asyncResult)
{
// Get the RequestState object from AsyncResult.
RequestState rs = (RequestState)asyncResult.AsyncState;
// Retrieve the ResponseStream that was set in RespCallback.
Stream responseStream = rs.ResponseStream;
// Read rs.BufferRead to verify that it contains data.
int read = responseStream.EndRead(asyncResult);
if (read > 0)
{
// Prepare a Char array buffer for converting to Unicode.
Char[] charBuffer = new Char[BUFFER_SIZE];
// Convert byte stream to Char array and then to String.
// len contains the number of characters converted to Unicode.
int len = rs.StreamDecode.GetChars(rs.BufferRead, 0, read, charBuffer, 0);
String str = new String(charBuffer, 0, len);
// Append the recently read data to the RequestData stringbuilder
// object contained in RequestState.
RequestData.Append(PureAnalyzer_WebApp.DoubleAt + rs.ResultName + PureAnalyzer_WebApp.DoubleColon);
// ESResultBE ESObj = new ESResultBE();
System.Web.Script.Serialization.JavaScriptSerializer JsonSerializer = new System.Web.Script.Serialization.JavaScriptSerializer();
ESResultBE ObjESResultBE;
string Result = "";
RequestData.Append(Encoding.ASCII.GetString(rs.BufferRead, 0, read).ToString());
// Continue reading data until
// responseStream.EndRead returns –1.
IAsyncResult ar = responseStream.BeginRead(
rs.BufferRead, 0, BUFFER_SIZE,
new AsyncCallback(ReadQtoRCallBack), rs);
}
else
{
// Close down the response stream.
responseStream.Close();
if (ResponseFlag[rs.ResultName] != "1")
{
ResponseFlag[rs.ResultName] = "1";
CompleteCount++;
}
// Set the ManualResetEvent so the main thread can exit.
allDone.Set();
}
return;
}
I'm running the following piece of code which uses a delegate to return an asynchronous Network Stream:
static void Main(string[] args)
{
NetworkStream myNetworkStream;
Socket socket;
IPEndPoint maxPort = new IPEndPoint(IPAddress.Parse("xxx.xxx.xxx.xxx"), xxxx);
socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.IP);
socket.Connect(maxPort);
myNetworkStream = new NetworkStream(socket);
byte[] buffer = new byte[1024];
int offset = 0;
int count = 1024;
string Command = "LOGIN,,,xxxx\n";
ASCIIEncoding encoder = new ASCIIEncoding();
myNetworkStream.BeginRead(buffer, offset, count, new AsyncCallback(OnBeginRead), myNetworkStream);
myNetworkStream.Write(encoder.GetBytes(Command), 0, encoder.GetByteCount(Command));
while (true) { }
}
public static void OnBeginRead(IAsyncResult ar)
{
NetworkStream ns = (NetworkStream)ar.AsyncState;
int bufferSize = 1024;
byte[] received = new byte[bufferSize];
ns.EndRead(ar);
int read;
while (true)
{
if (ns.DataAvailable)
{
string result = String.Empty;
read = ns.Read(received, 0, bufferSize);
result += Encoding.ASCII.GetString(received);
received = new byte[bufferSize];
result = result.Replace(" ", "");
result = result.Replace("\0", "");
result = result.Replace("\r\n", ",");
Console.WriteLine(result);
}
}
}
It works, but my CPU usage is through the roof (50% on an Intel Core i3), so obviously I'm doing it wrong, but how so?
Thanks
You're only reading the very first bytes asynchronously, afterwards you end up in an infinite loop with sync read operations in your OnBeginRead method (which is a confusing name BTW). At the same time, those first bytes are discarded in your current code.
You need to process the data after EndRead (which is a function returning how many bytes were read into the buffer in this async operation), and then start another async read with BeginRead and return (there is no looping in the async code!).
Edited to add a sample showing how async reading would work:
internal class StreamHelper {
private readonly NetworkStream stream;
private readonly byte[] buffer = new byte[1024];
public StreamHelper(Socket socket) {
stream = new NetworkStream(socket);
}
public NetworkStream Stream {
get {
return stream;
}
}
public byte[] Buffer {
get {
return buffer;
}
}
}
private static void Main(string[] args) {
IPEndPoint maxPort = new IPEndPoint(IPAddress.Parse("xxx.xxx.xxx.xxx"), 100);
Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.IP);
socket.Connect(maxPort);
StreamHelper helper = new StreamHelper(socket);
helper.Stream.BeginRead(helper.Buffer, 0, helper.Buffer.Length, StreamReadCallback, helper);
string Command = "LOGIN,,,xxxx\n";
byte[] bytes = Encoding.ASCII.GetBytes(Command);
// note: the write isn't async, but should maybe be converted as well
helper.Stream.Write(bytes, 0, bytes.Length);
Console.ReadLine(); // wait for a return key press
}
private static void StreamReadCallback(IAsyncResult ar) {
StreamHelper helper = (StreamHelper)ar.AsyncState;
// note: EndRead will throw an exception if something went wrong - you should deal with that
int bytesRead = helper.Stream.EndRead(ar);
if (bytesRead > 0) {
string charsRead = Encoding.ASCII.GetString(helper.Buffer, 0, bytesRead);
Console.Write(charsRead);
helper.Stream.BeginRead(helper.Buffer, 0, helper.Buffer.Length, StreamReadCallback, helper);
}
}
You are looping continuously on the main thread:
while (true) { }
This causes the CPU core of that thread to be at full capacity at all times. Try to sleep in order to prevent the thread from taking up CPU time unnecessarily:
while (true) { Thread.Sleep(5000); }
Perhaps replace the inefficiency of spinning the processor at the bottom of your main method from
while (true) { }
to
Console.ReadLine();
Incidentally Lucero is spot on. You're moving into an infinte loop (in OnBeginRead) with the thread that calls the callback method. This feels wrong. Callbacks should be dealt with asap to let the calling thread carry on processing. Normally you would extract the data in the callback and post a signal to your own thread to process the rest. Perhaps a TPL thread will help here.
I have some pretty simple code that reads lines from a network stream that it connects to. In the code example only one line is every read and it doesn't carry on getting more from the server.
What is wrong?
byte[] readBuffer = new byte[1024];
byte[] tempBuff = new byte[1024];
int tempBuffSize = 0;
private void btnConnect_Click(object sender, EventArgs e)
{
TcpClient tcpClient = new TcpClient("192.168.1.151", 5505);
NetworkStream stream = tcpClient.GetStream();
stream.BeginRead(readBuffer, 0, 1024, readHandler, tcpClient);
}
void readHandler(IAsyncResult result)
{
TcpClient tcpClient = (TcpClient)result.AsyncState;
int dataLen = tcpClient.GetStream().EndRead(result);
int currStart = 0;
int currEnd = -1;
for (int i = 0; i < dataLen; i++)
{
if (readBuffer[i] == '\r' && i < (readBuffer.Length - 1) &&
readBuffer[i + 1] == '\n')
{
// Set the end of the data
currEnd = i - 1;
// If we have left overs from previous runs:
if (tempBuffSize != 0)
{
byte[] joinedData = new byte[tempBuffSize + (currEnd - currStart + 1)];
Array.Copy(tempBuff, 0, joinedData, 0, tempBuffSize);
Array.Copy(readBuffer, currStart, joinedData, tempBuffSize, (currEnd - currStart + 1));
System.Text.Encoding enc = System.Text.Encoding.ASCII;
string myString = enc.GetString(joinedData);
System.Diagnostics.Debug.Write(myString);
tempBuffSize = 0;
}
else
{
System.Text.Encoding enc = System.Text.Encoding.ASCII;
string myString = enc.GetString(readBuffer);
System.Diagnostics.Debug.Write(myString);
// HandleData(readBuffer, currStart, currEnd);
}
// Set the new start - after our delimiter
currStart = i + 2;
}
}
// See if we still have any leftovers
if (currStart < dataLen)
{
Array.Copy(readBuffer, currStart, tempBuff, 0, dataLen - currStart);
tempBuffSize = dataLen - currStart;
}
}
Why do you expect it to read the whole information in the first place? I am not an expert but it seems to me that neither the synchronous nor the asynchronous methods guarantee reading all the data (whatever that means because as long as a socket is open more data can arrive). After the code in your EndRead method you should call Read or BeginRead again if you expect more data. You should know if more data is expected based on the protocol you've established with the client.
I have faced similar issues when i developed tcp apps for an embedded device. In my case the problem was the device was giving out data in a delayed time and hence before the rest of the data can come in the control moves to the next line in the program fetching only the initial data from the server. I got around this by introducing a delay.
Just after the line where you read data from the server introduce a delay and for that reason it would be better to run this on a separate thread
thread.sleep(3000)
This should be your problem most probably.
maybe your stream object got disposed when it got out of scope, before the readHandler could be called again. try promoting tcpClient and stream to class scope instead of method scope, or move the reading to a separate thread that exits when the operation has finished.
I try to use stream.DataAvailable to judge if it is finished,but sometimes the value is false but after a little while it is true again,i have to set a counter and judge the end by the symbol '>' like this
int connectCounter = 0;
while (connectCounter < 1200)
{
if (stream.DataAvailable)
{
while (stream.DataAvailable)
{
byte[] buffer = new byte[bufferSize];
int flag = stream.Read(buffer, 0, buffer.Length);
string strReadXML_t = System.Text.Encoding.Default.GetString(buffer);
strReadXML = strReadXML + strReadXML_t.Replace("\0", string.Empty);
}
if (strReadXML.Substring(strReadXML.Length - 1, 1).Equals(">"))
{
break;
}
}
Thread.Sleep(100);
connectCounter++;
}
is there any good methord to deal with it?Thank you!
You have a couple options. You can use a synchronous, blocking Read, or you can use an asynchronous IO pattern.
If you simply call stream.Read(), the call will block, and wait forever (until the TCP timeout), until data is available. It seems you don't want to do that. You want to wait, at most, 120 seconds (1200ms * 100), for the data to be completely read.
Something like this:
private class AsyncState
{
public NetworkStream ns;
public ManualResetEvent e;
public byte[] b;
public String strReadXML;
}
public void Run()
{
TcpClient client ;//= ...
NetworkStream networkStream = client.GetStream();
byte[] buffer = new byte[1024];
var completedEvent = new ManualResetEvent(false);
networkStream.BeginRead(buffer, 0, buffer.Length,
AsyncRead,
new AsyncState
{
b = buffer,
ns = networkStream,
e = completedEvent,
strReadXML = ""
});
// do other stuff here. ...
// finally, wait 120s for the reading to complete
bool success = completedEvent.WaitOne(1200*100, false);
if (!success)
{
client.Close();
}
}
private void AsyncRead(IAsyncResult ar)
{
AsyncState state = ar as AsyncState;
int n = state.ns.EndRead(ar);
if (n == 0)
{
// no more bytes to read
// signal completion
state.e.Set();
return;
}
// state.buffer now contains the bytes read
string s = System.Text.Encoding.Default.GetString(state.b);
state.strReadXML = state.strReadXML + s.Replace("\0", string.Empty);
if (state.strReadXML.EndsWith(">"))
{
// got the "end". signal completion
state.e.Set();
return;
}
// read again
state.ns.BeginRead(state.b, 0, state.b.Length, AsyncRead, state);
}
Try asynchronous reading.
When your callback is called, you can read the existing data buffer and then call
BeginRead again. So that when somre more data is received, you callback will be called again.
Something like:
void DataReceived(IAsyncResult result) ///I am not sure about the parameters.
{
///read data from buffer
networkstream.BeginRead(
buffer, 0, buffer.Length,
new AsyncCallback(DataReceived),
null);
}
I think this is a pretty decent approach.
Do you have control over the sending application? If you do you can change the sending application to wrap the NetworkStream in a BinaryWriter and use .Write( string ) which will send the length of the string, and then the string itself. On the receiving application (as above) you can wrap the NetworkStream in a BinaryReader and call .ReadString() which will read the length from the stream, then read the correct length string in for you.
If you don't have control over the sending application then you can check the return from the stream.Read() call as it returns 0 at the end of the stream - though this will require the sending application to have closed the socket.
Also, stream.Read() is not guaranteed to return the number of bytes you requested, it could return less. Your flag variable should really be bytesread, and you should then pass this bytesread variable to the .GetString method.