Develop Reference

ZMQ recv randomly picks up garbage data does any one know why?

I have a client server app which is sending data from a C# client to a C++ server. When the server receives this data request, 9 out of 10 times it works ok, but there is always 1 time were there will be garbage data appended to the end of the received data on the server side.
for example instead of receiving a number 1 it will receive 1C or 1#????
Here are snippets of the client and server code, any help will be appreciated.
C# client
int flagSide = 1;
msg = name;
msg += "+";
msg += "qty";
msg += "+";
msg += flagSide.ToString();
ZeroMQ.ZmqContext context = ZeroMQ.ZmqContext.Create();
ZeroMQ.ZmqSocket socket = context.CreateSocket(SocketType.REQ);
socket.Connect("tcp://111.111.0.111:5556");
socket.Send(Encoding.ASCII.GetBytes(msg.ToCharArray()));
Thread.Sleep(1);
string reply = socket.Receive(Encoding.ASCII);
Console.WriteLine("Received reply = " + reply + "\n");
C++ Server
std::tr1::unordered_map <std::string, std::string> aMap;
zmq::context_t context( 1 );
zmq::socket_t responder( context, ZMQ_REP );
responder.bind ("tcp://*:5556");
while ( 1 )
{
zmq::message_t recvMsg;
responder.recv( &recvMsg );
t = static_cast<char*>( recvMsg.data() );
std::string s(t);
std::vector<std::string> strs;
boost::split(strs, s, boost::is_any_of("+"));
aMap["name"] = strs[0];
aMap["qty"] = strs[1];
aMap["flag"] = strs[2];
..........
outputing the split string in the server reveals that sometimes the flag or strs[2] receives the garbage data.
Please help me if you see something that I'm not seeing.
Thanks

In C#, strings converted to bytes are not null-terminated, and c++ string expects a null terminated pointer.
So I presume what is happening here, is a buffer underflow. You are reading memory which does not belongs to the string.

C# Tcp is losing packets?

I wrote a C# chat software that uses a new (at least for me) system that I called request system. I don't know if that has been created before, but for now I think of it as my creation :P
Anyhow, this system works like this:
soc receives a signal
checks the signal
if the data it just received is the number 2, the client software knows that the server is about to send a chat message. if the number is 3, so the client knows that the server is about to send the member list, and so on.
The problem is this: when I do step-by-step in VS2012 it works fine, the chat is working properly. When I use it on debug mode or just run it on my desktop, there seems to be missing data, and it shouldn't be because the code is working just fine...
Example of code for the sending&receiving message on client:
public void RecieveSystem()
{
while (true)
{
byte[] req = new byte[1];
soc.Receive(req);
int requestID = int.Parse(Encoding.UTF8.GetString(req));
if (requestID == 3)
{
byte[] textSize = new byte[5];
soc.Receive(textSize);
byte[] text = new byte[int.Parse(Encoding.UTF8.GetString(textSize))];
soc.Receive(text);
Dispatcher.Invoke(() => { ChatBox.Text += Encoding.UTF8.GetString(text) + "\r\n"; });
}
}
}
public void OutSystem(string inputText)
{
byte[] req = Encoding.UTF8.GetBytes("3");
soc.Send(req);
byte[] textSize = Encoding.UTF8.GetBytes(Encoding.UTF8.GetByteCount(inputText).ToString());
soc.Send(textSize);
byte[] text = Encoding.UTF8.GetBytes(inputText);
soc.Send(text);
Thread.CurrentThread.Abort();
}
and on the server:
public void UpdateChat(string text)
{
byte[] req = Encoding.UTF8.GetBytes("3");
foreach (User user in onlineUsers)
user.UserSocket.Send(req);
byte[] textSize = Encoding.UTF8.GetBytes(Encoding.UTF8.GetByteCount(text).ToString());
foreach (User user in onlineUsers)
user.UserSocket.Send(textSize);
byte[] data = Encoding.UTF8.GetBytes(text);
foreach (User user in onlineUsers)
user.UserSocket.Send(data);
}
public void RequestSystem(Socket soc)
{
~~~
}
else if (request == 3)
{
byte[] dataSize = new byte[5];
soc.Receive(dataSize);
byte[] data = new byte[int.Parse(Encoding.UTF8.GetString(dataSize))];
soc.Receive(data);
UpdateChat(Encoding.UTF8.GetString(data));
}
}
catch
{
if (!soc.Connected)
{
Dispatcher.Invoke(() => { OnlineMembers.Items.Remove(decodedName + " - " + soc.RemoteEndPoint); Status.Text += soc.RemoteEndPoint + " Has disconnected"; });
onlineUsers.Remove(user);
Thread.CurrentThread.Abort();
}
}
}
}
What could be the problem?

You're assuming that you'll have one packet for each Send call. That's not stream-oriented - that's packet-oriented. You're sending multiple pieces of data which I suspect are coalesced into a single packet, and then you'll get them all in a single Receive call. (Even if there are multiple packets involved, a single Receive call could still receive all the data.)
If you're using TCP/IP, you should be thinking in a more stream-oriented fashion. I'd also encourage you to change the design of your protocol, which is odd to say the least. It's fine to use a length prefix before each message, but why would you want to encode it as text when you've got a perfectly good binary connection between the two computers?
I suggest you look at BinaryReader and BinaryWriter: use TcpClient and TcpListener rather than Socket (or at least use NetworkStream), and use the reader/writer pair to make it easier to read and write pieces of data (either payloads or primitives such as the length of messages). (BinaryWriter.Write(string) even performs the length-prefixing for you, which makes things a lot easier.)

Socket receives the correct bytes, bytes translate to empty string

I've been breaking my head over a bug in this system I've been building. Basically, I use sockets to communicate between two C# applications. Or rather a Unity C# script server and a C# client application.
With manual tests, the system works perfectly fine, no anomalies whatsoever.
In order to test performance and multi-user functionality, I wrote up a tester class which launches multiple threads(clients), and have those fire X amount of messages at the server. Here's where my problem occurs...Sometimes.
When a Socket sends or receives, it returns an integer container the amount of bytes that was sent/received. When the problem occurs, I can see that the correct amount of bytes arrived at the server. However, after putting the bytes into a string, suddenly I'm left with an empty string, instead of the message I'd normally see here.
I'm at a loss at to what's causing this problem. I'm using Encoding.Default.GetString() to translate the bytes into a string.
Any help is appreciated!
David
public void ReceiveFromClient (Socket handlerSocket)
{
serverBuffer = new byte[iBufferSize]; //iBufferSize = 8192;
int i = handlerSocket.Receive (serverBuffer);
Debug.Log ("Bytes received: " + i);
string message = Encoding.UTF8.GetString (serverBuffer, 0, i);
Debug.Log ("Message received: " + message);
//Do stuff with the message
}
bool SendMessageToUnity(string input)
{//returns a bool saying whether the message was sent or not
if (clientSocket != null)
{
if (clientSocket.Connected)
{
byte[] bytes = Encoding.UTF8.GetBytes(input+"|");
txtOutput.BeginInvoke(new Action(() => txtOutput.AppendText("Sending message: " + Encoding.UTF8.GetString(bytes) + Environment.NewLine)));
int i = clientSocket.Send(bytes);
txtOutput.BeginInvoke(new Action(() => txtOutput.AppendText("Sending "+i+" bytes. "+ Environment.NewLine)));
return true;
}
}
return false;
}

Look for for a zero value ('\0') in your array of bytes before converting it to a string.
private string GetString(byte[] data)
{
data = data.Where(b => b != 0).ToArray();
return Encoding.UTF8.GetString(data);
}

If you get the byte array correctly than the problem in the Encoding.
Check the sending Encoding usually UTF8 but you have to check it out.
and then var inputStr = Encoding.UTF8.GetString(InputByteArray);
^^

Incomplete data received across network from C# to Arduino

I am trying to send a word over to an Arduino running as a server, from a WPF C# application. Every now and again the complete work is not sent.
C# Code
public void send(String message)
{
TcpClient tcpclnt = new TcpClient();
ConState.Content = "Connecting.....";
try
{
tcpclnt.Connect("192.168.0.177", 23);
ConState.Content = "Connected";
String str = message;
Stream stm = tcpclnt.GetStream();
ASCIIEncoding asen = new ASCIIEncoding();
byte[] ba = asen.GetBytes(str);
stm.Write(ba, 0, ba.Length);
tcpclnt.Close();
}
catch (Exception)
{
ConState.Content = "Not Connected";
return;
}
}
How it is sent to the method:
String mes = "back;";
send(mes);
Arduino code:
if (client.available() > 0) {
// Read the bytes incoming from the client:
char thisChar = client.read();
if (thisChar == ';')
{
//Add a space
Serial.println("");
}
else {
//Print because it's not a space
Serial.write(thisChar);
}
}
The Arduino is using the chat server example. I am sending "back;" and "forward;" across. The results on the serial monitor:
back
forwaback
forward
back
forwaforwar

The problem seems to be with this code:
if (client.available() > 0) {
// read the bytes incoming from the client:
char thisChar = client.read();
...
}
What it does is:
Check if we have received data from the client
Read a single byte from the client buffer
Exit, and go on to do other things
As the OP pointed out, this comes direct from Arduino chat server example. In that example, this working correctly in loop() depends on the alreadyConnected flag being set right after a new connection is made: if it isn't, then the buffer is flushed before any data is read. That's one possible landmine.
Nonetheless, there is no reason to change the if block to be a while loop in the OP's case so, in other words instead of
if (client.available() > 0) {
have
while (client.available() > 0) {
The only reason to have an if statement there is to make sure that you frequently do other processing in loop() if you have clients that send a lot of data: If the reading of client data is done from inside a while this loop will not exit until the there is no more data from the client. Since this doesn't seem to be an issue in the asked-about case, the if to while change makes sense.

WebSockets in firefox

For implementing my websocket server in C# I'm using Alchemy framework. I'm stuck with this issue. In the method OnReceive when I try to deserialize json object, I get a FormatException:
"Incorrect format of the input string." (maybe it's different in english, but I'm getting a localized exception message and that's my translation :P). What is odd about this is that when I print out the context.DataFrame I get: 111872281.1341000479.1335108793.1335108793.1335108793.1; __ad which is a substring of the cookies sent by the browser: __gutp=entrystamp%3D1288455757%7Csid%3D65a51a83cbf86945d0fd994e15eb94f9%7Cstamp%3D1288456520%7Contime%3D155; __utma=111872281.1341000479.1335108793.1335108793.1335108793.1; __adtaily_ui=cupIiq90q9.
JS code:
// I'm really not doing anything more than this
var ws = new WebSocket("ws://localhost:8080");
C# code:
static void Main(string[] args) {
int port = 8080;
WebSocketServer wsServer = new WebSocketServer(port, IPAddress.Any) {
OnReceive = OnReceive,
OnSend = OnSend,
OnConnect = OnConnect,
OnConnected = OnConnected,
OnDisconnect = OnDisconnect,
TimeOut = new TimeSpan(0, 5, 0)
};
wsServer.Start();
Console.WriteLine("Server started listening on port: " + port + "...");
string command = string.Empty;
while (command != "exit") {
command = Console.ReadLine();
}
Console.WriteLine("Server stopped listening on port: " + port + "...");
wsServer.Stop();
Console.WriteLine("Server exits...");
}
public static void OnReceive(UserContext context) {
string json = "";
dynamic obj;
try {
json = context.DataFrame.ToString();
Console.WriteLine(json);
obj = JsonConvert.DeserializeObject(json);
} catch (Exception e) {
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
return;
}
}
On the C# side I'm using Newtonsoft.Json, though it's not a problem with this library...
EDIT:
One more thing - I browsed through the code in here: https://github.com/Olivine-Labs/Alchemy-Websockets-Example and found nothing - I mean, I'm doing everything the same way authors did in this tutorial...
EDIT:
I was testing the above code in Firefox v 17.0.1, and it didn't work, so I tested it under google chrome, and it works. So let me rephrase the question - what changes can be made in js, so that firefox would not send aforementioned string?

I ran into the same issue - simply replacing
var ws = new WebSocket("ws://localhost:8080");
with
var ws = new WebSocket("ws://127.0.0.1:8080");
fixed the issue for me.

In C# console app I connect the client to the server using :
var aClient = new WebSocketClient(#"ws://127.0.0.1:81/beef");
Your code above is connecting using
var ws = new WebSocket("ws://localhost:8080");
There could be one of two issues -
First is to see if WebSocketClient works instead.
To make sure your url is of the format ws://ur:port/context. This threw me off for a while.