Develop Reference

Replicate PuTTY's raw connection with sockets

I am working with a heat cell with which i can only communicate through Ethernet. When trying to connect to it using sockets, I am stuck on waiting for the Receive() method to end.
I've checked the connection using PuTTY in raw mode and it worked just fine, I was able to send and receive messages.
This led me to believe that I needed to use some kind of raw communication as well, hence why I tried to use SharpPCap and the like. Using this, I am able to read from the cell (although I am faced with a few issues that aren't related to this post).
However, since I'm not very experienced with networking, I was wondering if there was a way to obtain the same results as when I used PuTTY's raw mode but using only Sockets ?
I've come accross this question that was left unanswered, apart from the fact that the author was advised not to use SocketType.Raw.
Below is the example from MSDN documentation that I adapted for my tests. It is supposed to send a request and then listen for the answer.
static void Main(string[] args)
{
System.Net.IPAddress host = System.Net.IPAddress.Parse("10.0.0.3");
int port = 2049;
Socket socket = new Socket(SocketType.Stream, ProtocolType.Tcp);
socket.Connect(host, port);
if (socket.Connected)
{
string request = "99997¶1¶1\\r";
var byteSent = Encoding.Default.GetBytes(request);
var byteReceived = new byte[256];
socket.Send(byteSent, byteSent.Length, 0);
int bytes = 0;
do
{
bytes = socket.Receive(byteReceived, byteReceived.Length, 0); // this is the line I'm being stuck on
} while (bytes > 0);
Console.WriteLine($"Result : {Encoding.Default.GetString(byteReceived)}");
Console.ReadLine();
}
else
{
Console.WriteLine("Connection Failed");
Console.ReadLine();
}
}
UPDATE
I used WireShark to take a look at what was being sent from my application. Turns out the 'CR' and 'LF' werent sent correctly. But correcting this didn't solve my problem.
I can see the server's answer to my request on WireShark, but still I can't read anything using socket.Receive(). Could it be possible that the server is faster to answer than my machine is to start listening ? I tried setting up the socket.ReceiveTimeout property and put my send/receive instructions in a loop, but still nothing returning from the Receive() statement.

The socket is in blocking mode by default (cf Socket.Blocking). That mean that, for your case, blocking until 256 byte has been read (Without timeout by default Socket.ReceiveTimeout).
You may want to set the Blocking mode to false, or set a Receive timeout in order to not block indefinitly.
Alternatively, you may want also to use async method to avoid blocking the main thread.

Simple One Thread Socket - TCP Server

First, I don't know if Stackoverflow is the best site to post this kind of message, but I don't know another sites like this.
In oder to understand properly tcp programmation in C#, I decided to do all possible ways from scratch. Here is what I want to know (not in the right order:
- Simple One Thread Socket Server (this article)
- Simple Multiple Threads Socket Server (I don't know how, cause threads are complicated)
- Simple Thread Socket Server (put the client management in another thread)
- Multiple Threads Socket Server
- Using tcpListener
- Using async / Await
- Using tasks
The ultimate objective is to know how to do the best tcp server, without just copy/paste some parts of come, but understand properly all things.
So, this is my first part : a single thread tcp server.
There is my code, but I don't think anybody will correct something, because it's quite a copy from MSDN : http://msdn.microsoft.com/en-us/library/6y0e13d3(v=vs.110).aspx
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
namespace SimpleOneThreadSocket
{
public class ServerSocket
{
private int _iPport = -1;
private static int BUFFER_SIZE = 1024;
private Socket _listener = null;
public ServerSocket(int iPort)
{
// Create a TCP/IP socket.
this._listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
// Save the port
this._iPport = iPort;
}
public void Start()
{
byte[] buffer = null;
String sDatasReceived = null;
// Bind the socket to loopback address
try
{
this._listener.Bind(new System.Net.IPEndPoint(System.Net.IPAddress.Loopback, _iPport));
this._listener.Listen(2);
}
catch (Exception e)
{
System.Console.WriteLine(e.ToString());
}
// Listening
try
{
Console.WriteLine("Server listening on 127.0.0.1:" + _iPport);
while (true)
{
Socket client = this._listener.Accept();
Console.WriteLine("Incoming connection from : " + IPAddress.Parse(((IPEndPoint)client.RemoteEndPoint).Address.ToString()) + ":" + ((IPEndPoint)client.RemoteEndPoint).Port.ToString());
// An incoming connection needs to be processed.
while (true)
{
buffer = new byte[BUFFER_SIZE];
int bytesRec = client.Receive(buffer);
sDatasReceived += Encoding.ASCII.GetString(buffer, 0, bytesRec);
if (sDatasReceived.IndexOf("<EOF>") > -1)
{
// Show the data on the console.
Console.WriteLine("Text received : {0}", sDatasReceived);
// Echo the data back to the client.
byte[] msg = Encoding.ASCII.GetBytes(sDatasReceived);
client.Send(msg);
sDatasReceived = "";
buffer = null;
}
else if (sDatasReceived.IndexOf("exit") > -1)
{
client.Shutdown(SocketShutdown.Both);
client.Close();
sDatasReceived = "";
buffer = null;
break;
}
}
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
}
}
But I have some questions about that :
Listen Method from Socket have a parameter : backlog. According to MSDN, backlog is the number of available connection. I don't know why, when I put 0, I can connect to my server with multiple Telnet sessions. EDIT : 0 & 1 both allow 2 connections (1 current, 1 pending), 2 allow 3 connections (1 current, 2 pending), etc... So I didn't understand well the meaning of MSDN.
Can you confirm that Accept Method will take each connection one after one, that's why I see text from differents Telnet session in my server ?
Can you confirm (my server is a C# library) I can't kill my server (with this kind of code) without killing the process ? It could be possible with threads but it will come later.
If something is wrong in my code, please help me :)
I will come back soon with a simple multiple thread socket server, but I don't know how (I think one step is available before using threads or async/await).

First off, do your best not to even learn this. If you can possibly use a SignalR server, then do so. There is no such thing as a "simple" socket server at the TCP/IP level.
If you insist on the painful route (i.e., learning proper TCP/IP server design), then there's a lot to learn. First, the MSDN examples are notoriously bad starting points; they barely work and tend to not handle any kind of error conditions, which is absolutely necessary in the real world when working at the TCP/IP level. Think of them as examples of how to call the methods, not examples of socket clients or servers.
I have a TCP/IP FAQ that may help you, including a description of the backlog parameter. This is how many connections the OS will accept on your behalf before your code gets around to accepting them, and it's only a hint anyway.
To answer your other questions: A single call to Accept will accept a single new socket connection. The code as-written has an infinite loop, so it will work like any other infinite loop; it will continue executing until it encounters an exception or its thread is aborted (which happens on process shutdown).
If something is wrong in my code, please help me
Oh, yes. There are lots of things wrong with this code. It's an MSDN socket example, after all. :) Off the top of my head:
The buffer size is an arbitrary value, rather low. I would start at 8K myself, so it's possible to get a full Ethernet packet in a single read.
The Bind explicitly uses the loopback address. OK for playing around, I guess, but remember to set this to IPAddress.Any in the real world.
backlog parameter is OK for testing, but should be int.MaxValue on a true server to enable the dynamic backlog in modern server OSes.
Code will fall through the first catch and attempt to Accept after a Bind/Listen failed.
If any exception occurs (e.g., from Listen or Receive), then the entire server shuts down. Note that a client socket being terminated will result in an exception that should be logged/ignored, but it would stop this server.
The read buffer is re-allocated on each time through the loop, even though the old buffer is never used again.
ASCII is a lossy encoding.
If a client cleanly shuts down without sending <EOF>, then the server enters an infinite busy loop.
Received data is not properly separated into messages; it is possible that the echoed message contains all of one message and part of another. In this particular example it doesn't matter (since it's just an echo server and it's using ASCII instead of a real encoding), but this example hides the fact that you need to handle message framing properly in any real-world application.
The decoding should be done after the message framing. This isn't necessary for ASCII (a lossy encoding), but it's required for any real encodings like UTF8.
Since the server is only either receiving or sending at any time (and never both), it cannot detect or recover from a half-open socket situation. A half-open socket will cause this server to hang.
The server is only capable of a single connection at a time.
That was just after a brief readthrough. There could easily be more.

c# sockets blocking

Say I have a TcpClient that I accept from a TcpServer in c#, and for some reason it keeps defaulting with blocking off. Is there some other force that can change how the socket blocking is set? Like say is it affected by the remote connection at all?
I know I set blocking to fale a few builds back, but I changed it, and even introduced the TcpClient instead of a straight socket. I haven't specifically changed the blocking back to true, I just commented blocking = false out. Is that something that persists maybe with the endpoint?
I don't know though it just seemed that as I was programming one day my sockets just became unruley without any real change in their code.
public void GetAcceptedSocket(TcpClient s)
{
try
{
sock = s;
IPEndPoint remoteIpEndPoint = s.Client.RemoteEndPoint as IPEndPoint;
strHost = remoteIpEndPoint.Address.ToString();
intPort = remoteIpEndPoint.Port;
ipEnd = remoteIpEndPoint;
sock.ReceiveTimeout = 10000;
boolConnected = true;
intLastPing = 0;
LastPingSent = DateTime.Now;
LastPingRecieved = DateTime.Now;
handleConnect(strHost, intPort);
oThread = new Thread(new ThreadStart(this.run));
oThread.Start();
}
catch (Exception e)
{
handleError(e, "Connect method: " + e.Message);
}
}

Like say is it affected by the remote
connection at all?
Nope.
It would have been better if you show some code where you create Socket or TcpClient in server side. I cannot find TcpServer class in C#. Are you talking about TcpListener?
If you are, please make it sure that you set Socket's Blocking true if you use AcceptSocket to create a new socket. If you call AcceptTcpClient, you should not worry about blocking or non-blocking mode as TcpClient is always blocking mode.
The TcpClient class provides simple
methods for connecting, sending, and
receiving stream data over a network
in synchronous blocking mode.
from MSDN.

There really is no answer, I've never duplicated this, and after a few months, and a fresh windows install, picked the code up, and didn't have the problem anymore. Very weird though, defiantly couldn't find a programmatic reasoning for it happening. Like I said, after a few months it just kinda worked?
They should really have an 'Unawnserable' option :p.

How do I properly simulate high latency on a C# socket server test?

I have a class that uses 'System.Net.Sockets.Socket' directly for network comunication, and currently I use the following two interfaces to break dependency on the Socket class :
public interface ISocketListener
{
void Listen(int backlog);
ISocket Accept();
void Bind(EndPoint endpoint);
}
public interface ISocket
{
void Connect (EndPoint ep);
Stream CommunicationStream { get; }
void Close();
void Close(int timeout);
void Shutdown();
}
In the production implementation I just redirect all the method calls to the private Socket object. In testing environment I use a MemoryStream as the comunication 'pipe' between the sockets.
Since I have little experience in the subject, some questions appeared in my head while writing tests : Are there any 'formal' good practices in testing this kind of software? When doing integration testing, how do I test the performance of this server in multiple connections/high latency situations (more specifically, how to simulate these situations)? Why there are asynchronous versions of Socket.Accept/Socket.Receive? Can I replace the asynchronous methods for handling their synchronous versions in separate threads?

You could simulate high latency by creating a proxy server to sit inbetween your connection, then you can just add a delay when re-sending the data.

I'm not familiar with the socket object. However, I have done some research about testing. It looks like you're on the right track. Writing interfaces that can refer to either the actual objects that will be used in production or test objects is generally considered good practice. This pattern is known as "Dependency Injection".
I don't know how the socket works, but what you can do for testing purposes is create a new test object to use in place of the socket you want to test. This test object can simply call Thread.Sleep(x) to add some simulated latency before returning data.
Edit: I would also like to point out that you're going to have to be very careful to ascertain exactly where the network latency occurs in your code so that when you inject the artificial latency, you add it to the same place in the code where it would occur in the real world.

Are there any 'formal' good practices
in testing this kind of software?
I can speak to this but I'm sure there is some good information here on stack or out in the wild.
When doing integration testing, how
do I test the performance of this
server in multiple connections/high
latency situations (more specifically,
how to simulate these situations)?
For a simple start write some test and logging routines, running them on a separate box, that hammered the server with a verity of requests...including nonsensical ones. Running log4net on both would also be helpful. Introducing some lag could be done, as already mentioned, via a sort of proxy...a box that sits between the client and server. The client points to the proxy and the proxy modifies the data...delay, change packet order, etc....sends to to the server...reverse and repeat. As a slight aside...I would avoid Thread.Sleep(...). Better to use something like this :
lock (timelock) { Monitor.Wait(timelock, TimeoutInMilliseconds); }
...asynchronous versions of
Socket.Accept/Socket.Receive? Can I
replace the asynchronous methods for
handling their synchronous versions in
separate threads?
You have many options, asynchronous methods, plain old threading, the threadpool, the TPL, etc. The best choice is application specific.
Here is an Asynchronous Server Socket Example from the MSDN entry on sockets.
As luck would have it....the O'Reilly book: C# 4.0 in a Nutshell has an excellent set of TCP Server examples that cover both asynchronous/threaded methods in blocking/non-blocking manifestations. Here is one of the examples....
public class Server
{
public void Serve(IPAddress address, int port)
{
TcpListener listener = new TcpListener(address, port);
listener.Start();
while (true)
{
TcpClient c = listener.AcceptTcpClient();
Task.Factory.StartNew(Accept, c);
}
}
void Accept(object clientObject)
{
using (TcpClient client = (TcpClient)clientObject)
{
using (NetworkStream n = client.GetStream())
{
byte[] data = new byte[5000];
int bytesRead = 0; int chunkSize = 1;
while (bytesRead < data.Length && chunkSize > 0)
{
bytesRead +=
chunkSize = n.Read /// Read is blocking
(data, bytesRead, data.Length - bytesRead);
}
Array.Reverse(data);
n.Write /// Write is blocking
(data, 0, data.Length);
}
}
}
}
Hope this is helpful.
P.S. Get a copy of C# 4.0 in a Nutshell...tis good.

~1 second TcpListener Pending()/AcceptTcpClient() lag

Probably just watch this video: http://screencast.com/t/OWE1OWVkO
As you see, the delay between a connection being initiated (via telnet or firefox) and my program first getting word of it.
Here's the code that waits for the connection
public IDLServer(System.Net.IPAddress addr,int port)
{
Listener = new TcpListener(addr, port);
Listener.Server.NoDelay = true;//I added this just for testing, it has no impact
Listener.Start();
ConnectionThread = new Thread(ConnectionListener);
ConnectionThread.Start();
}
private void ConnectionListener()
{
while (Running)
{
while (Listener.Pending() == false) { System.Threading.Thread.Sleep(1); }//this is the part with the lag
Console.WriteLine("Client available");//from this point on everything runs perfectly fast
TcpClient cl = Listener.AcceptTcpClient();
Thread proct = new Thread(new ParameterizedThreadStart(InstanceHandler));
proct.Start(cl);
}
}
(I was having some trouble getting the code into a code block)
I've tried a couple different things, could it be I'm using TcpClient/Listener instead of a raw Socket object? It's not a mandatory TCP overhead I know, and I've tried running everything in the same thread, etc.

Maybe it's some kind of dns resolve? Are you using IP address to access your server's host or some name which is being resolved by your DNS? The code ParmesanCodice gave should work with no delay unless there's something wrong on client/network side.
Try to add following line to your windows\system32\drivers\etc\hosts:
127.0.0.1 localhost
it may solve your problem or just connect as 127.0.0.1:85

You should consider accepting your clients asynchronously, this will most likely remove the lag you are seeing.
I've modified your code slightly
public IDLServer(System.Net.IPAddress addr,int port)
{
Listener = new TcpListener(addr, port);
Listener.Start();
// Use the BeginXXXX Pattern to accept clients asynchronously
listener.BeginAcceptTcpClient(this.OnAcceptConnection, listener);
}
private void OnAcceptConnection(IAsyncResult asyn)
{
// Get the listener that handles the client request.
TcpListener listener = (TcpListener) asyn.AsyncState;
// Get the newly connected TcpClient
TcpClient client = listener.EndAcceptTcpClient(asyn);
// Start the client work
Thread proct = new Thread(new ParameterizedThreadStart(InstanceHandler));
proct.Start(client);
// Issue another connect, only do this if you want to handle multiple clients
listener.BeginAcceptTcpClient(this.OnAcceptConnection, listener);
}

Doesn't the debugger add overhead ?
I had issues like this when I was building my MMO server.
can't remember how I got round it now.
I think this has something to do with resource allocation on services, I use the approach suggested by ParmesanCodice (well a similar one at least) and during testing I found that the first 5 to 10 connections were rubbish but after that the service seems to hammmer out new connections like theres no tomorrow ...
Maybe its a socket thing in the framework.
Have you tried a load test?
Throw say 1000 connections at it and see what happens, it should get faster after handling each one.

You could avoid the entire Listener.Pending while loop. AcceptTcpClient() is a blocking call so you could just let your code run and pend on that. I don't know why that loop would take 1 second (instead of 1 millisecond) but since you indicate that is where the lag is, you can get rid of it.