I'm attempting to learn ZeroMq for project at work although my background is in C#, and in the most simplest of tests I seem to have an issue where the socket.recv(...) call will block for the first received message, but after this throws an exception because the amount of data received is -1.
Currently my 'server' is:
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_REP);
socket.bind("tcp://127.0.0.1:5555");
while (true)
{
zmq::message_t message;
if (socket.recv(&message))
{
auto str = std::string(static_cast<char*>(message.data()), message.size());
printf("Receieved: %s\n", str.c_str());
}
}
This is basically from following the first example server within the ZeroMq documentation.
I'm pushing 1 bit of data from a C# 'client' using this code:
using (var context = new ZContext())
using (var requester = new ZSocket(context, ZSocketType.REQ))
{
requester.Connect(#"tcp://127.0.0.1:5555");
requester.Send(new ZFrame(#"hello"));
requester.Disconnect(#"tcp://127.0.0.1:5555");
}
Now I start the server, then start the client. I correctly receive the first message and I am correctly able to print this.
But now when I hit socket.recv(&message) again the code won't block but will instead throw an exception because the underlying zmq_msg_recv(...) returns a value of -1.
I'm unsure why this is occurring, I cannot see why it is expecting another message as I know that there is nothing else on this port. The only thing I came across is calling zmq_msg_close(...) but this should be called as part of the message_t destructor, which I have confirmed.
Is there anything I'm doing wrong in terms of the socket setup or how I'm using it for the recv(...) call to stop blocking?
Your problem is that you cannot receive 2 requests in a row with the REQ-REP pattern.
In the Request-Reply Pattern each request demands a reply. Your client needs to block until it receives a reply to its first request. Also, your server needs to reply to the requests before it services a new request.
Here is a quote referring to your exact issue from the guide.
The REQ-REP socket pair is in lockstep. The client issues zmq_send()
and then zmq_recv(), in a loop (or once if that's all it needs). Doing
any other sequence (e.g., sending two messages in a row) will result
in a return code of -1 from the send or recv call. Similarly, the
service issues zmq_recv() and then zmq_send() in that order, as often
as it needs to.
Related
I've been struggling to debug an issue with my C# socket application. The application is part of a university assignment that I am working on. The scope of the assignment is to build a webserver and game/application server using raw sockets. In my case the webserver serves static files and acts as a proxy for the game server. Serving files seems to be working well, but when forwarding requests through to the game server, there is a data transfer issue.
The complete source code is available on GitHub for reference. But I think the problematic area is in this area which tries to read the body of the response from the game server responding to the webserver's request. Here is the relevant code:
if (header.ContainsKey("content-length"))
{
var bodyLength = Convert.ToInt32(header["content-length"]);
Console.WriteLine($"Receiving body from game server. Expecting {bodyLength} bytes");
body = ReceiveBodyData(socket, bodyLength);
Console.WriteLine($"Finished receiving body from game server. Received {body.Length} bytes.");
}
When it is executing this code the first message is written to the console, but the second message never prints because the method ReceiveBodyData never returns (ends up in an infinite loop trying to read the body data).
Examining the output from the game server, I see that the connection is closed from it's end, but I'm not sure if that kills the recipient's ability to read data or not.
Can anyone assist in debugging this issue, careful to recognise that this is my assignment, so ideally please don't write tons of code.
I think I have found the issue. I have committed changes that fix two bugs in my code.
First bug
Sure enough, the ReceiveBodyData method had a logic flaw in it. The method was as follows:
protected byte[] ReceiveBodyData(Socket handler, int bodyLength)
{
// we know how many bytes to expect, so we create an appropriate array and keep track
// of how many bytes we've received.
var result = new byte[bodyLength];
var totalBytesReceived = 0;
while (true)
{
// The bug is in the line below
var bytesCount = handler.Receive(result, totalBytesReceived, bodyLength - totalBytesReceived, SocketFlags.None);
if (totalBytesReceived >= bodyLength)
{
return result;
}
}
}
I was assigning the number of bytes read to a variable bytesCount but I wasn't updating the value of totalBytesReceived so I was never meeting the condition to exit the loop.
Second bug
I don't quite understand why this bug occurs. Sometimes, after a valid connection is made and the request and response are exchanged, a second connection is created. My code gets stuck because it is listening for the request data to be transmitted, but it never is, because it doesn't seem to be a valid connection.
I don't know if the browser is trying to maintain a persistent connection or quite what is going on here. So I now return null if there is no data to receive. The code that listens for connections, just goes back to listening.
I am attempting to read the output of an IP phone system. My current implementation is using Streamreader.ReadAsync() to read a single char at a time from the stream into a result string, and then when I hit the custom line terminator provided by the phone system I am parsing the result string and storing the data. Here is a sample:
using (var monitor = new TcpClient(ip, port))
{
try
{
using (var reader = new StreamReader(monitor.GetStream()))
{
var streamChar = new char[1];
while (true)
{
await reader.ReadAsync(streamChar, 0, streamChar.Length);
What I am finding is that this implementation works properly during the day, but soon after 5PM I stop logging data. I am spitting out the read content to a file and I see no difference in the raw data provided at the time that logging ceases. When the ReadAsync() method hangs, I still see an open socket using netstat. I tested tonight and confirmed that if I simply allow the app to create a new TcpClient connecting to the same source within the same instance of the logging application, it begins receiving data normally.
I have previously tried using reader.EndOfStream and reader.Peek() to determine when to read the next char, but switched to this implementation because my understanding is that EndOfStream and Peek are blocking operations. I had seen the app hang at these conditionals, so I got rid of them and replaced them with what you see based on the understanding that this IP phone system will perpetually send data when new calls are received.
I am reaching out to see if anyone can confirm whether my implementation is flawed or whether I need to start focusing more on the phone system itself. Thank you for your help.
I decided to compensate for the timeout by writing back a single byte to the endpoint at a periodic interval. This resolved the issue and the connection is now remaining open. I had to deal with the response provided by the endpoint, but it would only reply with old call data that I simply discard.
First, I don't know if Stackoverflow is the best site to post this kind of message, but I don't know another sites like this.
In oder to understand properly tcp programmation in C#, I decided to do all possible ways from scratch. Here is what I want to know (not in the right order:
- Simple One Thread Socket Server (this article)
- Simple Multiple Threads Socket Server (I don't know how, cause threads are complicated)
- Simple Thread Socket Server (put the client management in another thread)
- Multiple Threads Socket Server
- Using tcpListener
- Using async / Await
- Using tasks
The ultimate objective is to know how to do the best tcp server, without just copy/paste some parts of come, but understand properly all things.
So, this is my first part : a single thread tcp server.
There is my code, but I don't think anybody will correct something, because it's quite a copy from MSDN : http://msdn.microsoft.com/en-us/library/6y0e13d3(v=vs.110).aspx
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
namespace SimpleOneThreadSocket
{
public class ServerSocket
{
private int _iPport = -1;
private static int BUFFER_SIZE = 1024;
private Socket _listener = null;
public ServerSocket(int iPort)
{
// Create a TCP/IP socket.
this._listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
// Save the port
this._iPport = iPort;
}
public void Start()
{
byte[] buffer = null;
String sDatasReceived = null;
// Bind the socket to loopback address
try
{
this._listener.Bind(new System.Net.IPEndPoint(System.Net.IPAddress.Loopback, _iPport));
this._listener.Listen(2);
}
catch (Exception e)
{
System.Console.WriteLine(e.ToString());
}
// Listening
try
{
Console.WriteLine("Server listening on 127.0.0.1:" + _iPport);
while (true)
{
Socket client = this._listener.Accept();
Console.WriteLine("Incoming connection from : " + IPAddress.Parse(((IPEndPoint)client.RemoteEndPoint).Address.ToString()) + ":" + ((IPEndPoint)client.RemoteEndPoint).Port.ToString());
// An incoming connection needs to be processed.
while (true)
{
buffer = new byte[BUFFER_SIZE];
int bytesRec = client.Receive(buffer);
sDatasReceived += Encoding.ASCII.GetString(buffer, 0, bytesRec);
if (sDatasReceived.IndexOf("<EOF>") > -1)
{
// Show the data on the console.
Console.WriteLine("Text received : {0}", sDatasReceived);
// Echo the data back to the client.
byte[] msg = Encoding.ASCII.GetBytes(sDatasReceived);
client.Send(msg);
sDatasReceived = "";
buffer = null;
}
else if (sDatasReceived.IndexOf("exit") > -1)
{
client.Shutdown(SocketShutdown.Both);
client.Close();
sDatasReceived = "";
buffer = null;
break;
}
}
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
}
}
But I have some questions about that :
Listen Method from Socket have a parameter : backlog. According to MSDN, backlog is the number of available connection. I don't know why, when I put 0, I can connect to my server with multiple Telnet sessions. EDIT : 0 & 1 both allow 2 connections (1 current, 1 pending), 2 allow 3 connections (1 current, 2 pending), etc... So I didn't understand well the meaning of MSDN.
Can you confirm that Accept Method will take each connection one after one, that's why I see text from differents Telnet session in my server ?
Can you confirm (my server is a C# library) I can't kill my server (with this kind of code) without killing the process ? It could be possible with threads but it will come later.
If something is wrong in my code, please help me :)
I will come back soon with a simple multiple thread socket server, but I don't know how (I think one step is available before using threads or async/await).
First off, do your best not to even learn this. If you can possibly use a SignalR server, then do so. There is no such thing as a "simple" socket server at the TCP/IP level.
If you insist on the painful route (i.e., learning proper TCP/IP server design), then there's a lot to learn. First, the MSDN examples are notoriously bad starting points; they barely work and tend to not handle any kind of error conditions, which is absolutely necessary in the real world when working at the TCP/IP level. Think of them as examples of how to call the methods, not examples of socket clients or servers.
I have a TCP/IP FAQ that may help you, including a description of the backlog parameter. This is how many connections the OS will accept on your behalf before your code gets around to accepting them, and it's only a hint anyway.
To answer your other questions: A single call to Accept will accept a single new socket connection. The code as-written has an infinite loop, so it will work like any other infinite loop; it will continue executing until it encounters an exception or its thread is aborted (which happens on process shutdown).
If something is wrong in my code, please help me
Oh, yes. There are lots of things wrong with this code. It's an MSDN socket example, after all. :) Off the top of my head:
The buffer size is an arbitrary value, rather low. I would start at 8K myself, so it's possible to get a full Ethernet packet in a single read.
The Bind explicitly uses the loopback address. OK for playing around, I guess, but remember to set this to IPAddress.Any in the real world.
backlog parameter is OK for testing, but should be int.MaxValue on a true server to enable the dynamic backlog in modern server OSes.
Code will fall through the first catch and attempt to Accept after a Bind/Listen failed.
If any exception occurs (e.g., from Listen or Receive), then the entire server shuts down. Note that a client socket being terminated will result in an exception that should be logged/ignored, but it would stop this server.
The read buffer is re-allocated on each time through the loop, even though the old buffer is never used again.
ASCII is a lossy encoding.
If a client cleanly shuts down without sending <EOF>, then the server enters an infinite busy loop.
Received data is not properly separated into messages; it is possible that the echoed message contains all of one message and part of another. In this particular example it doesn't matter (since it's just an echo server and it's using ASCII instead of a real encoding), but this example hides the fact that you need to handle message framing properly in any real-world application.
The decoding should be done after the message framing. This isn't necessary for ASCII (a lossy encoding), but it's required for any real encodings like UTF8.
Since the server is only either receiving or sending at any time (and never both), it cannot detect or recover from a half-open socket situation. A half-open socket will cause this server to hang.
The server is only capable of a single connection at a time.
That was just after a brief readthrough. There could easily be more.
I've tried checking the server:port with telnet and I'm getting the expected results. So either writer.Write() or reader.ReadLine() isn't working cause I get nothing from the server.
TcpClient socket = new TcpClient(hostname, port);
if (!socket.Connected) {
Console.WriteLine("Failed to connect!");
return;
}
TextReader reader = new StreamReader(socket.GetStream());
TextWriter writer = new StreamWriter(socket.GetStream());
writer.Write("PING");
writer.Flush();
String line = null;
while ((line = reader.ReadLine()) != null) {
Console.WriteLine(line);
}
Console.WriteLine("done");
EDIT: I might have found the issue. This code was based off examples I found on the web. I tried another irc server: open.ircnet.net:6669 and I got a response:
:openirc.snt.utwente.nl 020 * :Please wait while we process your connection.
It seems as if I probably need to run the reader in a Thread so it can just constantly wait for a response. However it does seem weird that the program got caught on the while loop without ever printing done to the console.
I think you need to provide further details. I'm just going to assume that because you can easily telnet to the server using the same port your problem lies in the evaluation of the Connected property...
if (!socket.Connected) {
Console.WriteLine("Failed to connect!");
return;
}
this is wrong because Microsoft clearly specifies in the documentation that the Connected property is not reliable
Because the Connected property only reflects the state of the connection as of the most recent operation, you should attempt to send or receive a message to determine the current state. After the message send fails, this property no longer returns true. Note that this behavior is by design. You cannot reliably test the state of the connection because, in the time between the test and a send/receive, the connection could have been lost. Your code should assume the socket is connected, and gracefully handle failed transmissions.
That said, you should not use this property to determine the state of the connection. Needless to say that using this property to control the flow of your console app will result in unexpected results.
Suggestion
Remove the evaluation of the Connected property
Wrap your GetStream and Write method calls in a try/catch block to handle network communication errors
reader.ReadLine() will just wait for any data to arrive. If no data arrive, it seems to hang. That's a feature of tcp (I don't like it either). You need to find out how the end of the message is defined and stop based on that end criterion. Be careful, the end of message identifier may be split into two or more lines...
RFC for ping says that the server may not respond to it & such connections has to be closed after a time. Please check the RFC: https://www.rfc-editor.org/rfc/rfc1459#section-4.6.2
I'm writing a message layer for my distributed system. I'm using IOCP, ie the Socket.XXXAsync methods.
Here's something pretty close to what I'm doing (in fact, my receive function is based on his):
http://vadmyst.blogspot.com/2008/05/sample-code-for-tcp-server-using.html
What I've found now is that at the start of the program (two test servers talking to each other) I each time get a number of SAEA objects where the .Buffer is entirely filled with zeroes, yet the .BytesTransferred is the size of the buffer (1024 in my case).
What does this mean? Is there a special condition I need to check for? My system interprets this as an incomplete message and moves on, but I'm wondering if I'm actually missing some data. I was under the impression that if nothing was being received, you'd not get a callback. In any case, I can see in WireShark that there aren't any zero-length packets coming in.
I've found the following when I Googled it, but I'm not sure my problem is the same:
http://social.msdn.microsoft.com/Forums/en-US/ncl/thread/40fe397c-b1da-428e-a355-ee5a6b0b4d2c
http://go4answers.webhost4life.com/Example/socketasynceventargs-buffer-not-ready-121918.aspx
I am sure not what is going on in the linked example. It appears to be using asynchronous sockets in a synchronous way. I cannot see any callbacks or similar in the code. You may need to rethink whether you need synchronous or asynchronous sockets :).
To the problem at hand stems from the possibility that your functions are trying to read/write to the buffer before the network transmit/receive has been completed. Try using the callback functionality included in the async Socket. E.g.
// This goes into your accept function, to begin receiving the data
socketName.BeginReceive(yourbuffer, 0, yourbuffer.Length,
SocketFlags.None, new AsyncCallback(OnRecieveData), socketName);
// In your callback function you know that the socket has finished receiving data
// This callback will fire when the receive is complete.
private void OnRecieveData(IAsyncResult input) {
Socket inSocket = (Socket)input.AsyncState; // This is just a typecast
inSocket.EndReceive(input);
// Pull the data out of the socket as you already have before.
// state.Data.Write ......
}