Develop Reference

c # punch NAT network with TCP / IP protocol

Punching the UDP network works and works. However, when it comes to TCP, it's possible that I'm writing something wrong but I'm not a beginner programmer, or maybe I don't understand something.
Of course, I'll shorten a bit, we assume that we already have something as trivial as connection to an external server :)
Packet synchronization between threads as well as their creation, queuing, serialization, sending raw bytes and much more I skip because it is about the element of creating TCP sockets and not what works.
We will use the TcpListener class for the server.
public void InitializeServer(IPAddress address, int port)
{
try
{
// 127.0.0.1 accept only local connections, 0.0.0.0 is open for whole internet connections
listener = new TcpListener(address, port);
socket = listener.Server;
// Enable NAT Translation
listener.AllowNatTraversal(true);
// Start listening for example 10 client requests.
listener.Start(listenQueue);
Debug.Log($"[L{socket.LocalEndPoint}]Server start... ", EDebugLvl.Log);
OnServerInitialize(true);
// Enter the listening loop.
StartListener();
}
catch (SocketException e)
{
Debug.LogError($"SocketException: {e}", EDebugLvl.Error);
OnServerInitialize(false);
}
}
We start listening
private void StartListener()
{
Debug.Log("\nWaiting for a connection... ");
listener.BeginAcceptTcpClient(AcceptCallback, listener);
}
When the server receives the connection, we create a new socket
private void AcceptCallback(IAsyncResult ar)
{
TcpListener server = (TcpListener)ar.AsyncState;
TcpClient newClient = null;
try
{
newClient = server.EndAcceptTcpClient(ar);
}
catch (Exception e)
{
Debug.LogError(e.ToString());
}
if (newClient != null && newClient.Connected)
{
//...
client.StartRead();
}
//Loop
StartListener();
}
We create a new socket at the client and try to establish a connection
public void Connect(IPEndPoint remote, IPEndPoint bind = null, bool reuseAddress = false)
{
if (bind == null)
{
client = new TcpClient();
}
else
{
client = new TcpClient(bind);
}
socket = client.Client;
if (reuseAddress)
{
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, reuseAddress);
//It throws me a error SocketOption so im comment this.
//socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseUnicastPort, reuseAddress);
}
client.BeginConnect(remote.Address, remote.Port, ConnectCallback, null);
}
The connection works without any problems and data transfer.
Unfortunately, here as we know, we must start a new socket listening on the same address and port that was created when connecting to the server. I do this for every client.
public void StartHost(Client server)
{
if (server != null && server.socket.Connected)
{
IPEndPoint localHost = (IPEndPoint)server.socket.LocalEndPoint;
InitializeHost(localHost.Address, localHost.Port);
}
}
public void InitializeHost(IPAddress address, int port, bool reuse = false)
{
try
{
listener = new TcpListener(address, port);
socket = listener.Server;
if (reuse)
{
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseUnicastPort, true);
}
// Enable NAT Translation
listener.AllowNatTraversal(true);
// Start listening for example 10 client requests.
listener.Start(listenQueue);
Debug.Log($"\n[L{socket.LocalEndPoint}]Host start... ", EDebugLvl.Log);
OnServerInitialize(true);
// Enter the listening loop.
StartListener();
}
catch (SocketException e)
{
Debug.LogError($"SocketException: {e}", EDebugLvl.Error);
OnServerInitialize(false);
}
}
private void StartListener()
{
Debug.Log("\nWaiting for a connection... ");
listener.BeginAcceptTcpClient(AcceptCallback, listener);
}
private void AcceptCallback(IAsyncResult ar)
{
TcpListener server = (TcpListener)ar.AsyncState;
TcpClient newClient = null;
try
{
newClient = server.EndAcceptTcpClient(ar);
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
if (newClient != null && newClient.Connected)
{
//...
client.StartRead();
}
//Loop
StartListener();
}
So, as they write everywhere ... client "B" sends a packet to the server that wants to establish a connection, the server sends information to the client "A" about the client "B" and vice versa
Then they both try to connect with the new socket? No problem...
public void Connect(IPEndPoint remote, IPEndPoint bind = null, bool reuseAddress = false)
{
if (bind == null)
{
client = new TcpClient();
}
else
{
client = new TcpClient(bind);
}
socket = client.Client;
if (reuseAddress)
{
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, reuseAddress);
//socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseUnicastPort, reuseAddress);
}
client.BeginConnect(remote.Address, remote.Port, ConnectCallback, null);
}
private void ConnectCallback(IAsyncResult ar)
{
try
{
client.EndConnect(ar);
}
catch (Exception e)
{
Debug.LogError(e.ToString(), EDebugLvl.ConnectionError);
}
if (client.Connected)
{
Debug.Log($"[P{socket.RemoteEndPoint}, L{socket.LocalEndPoint}]Connected", EDebugLvl.ConnectionLog);
stream = new NetworkStream(socket, FileAccess.ReadWrite, true);
StartRead();
}
ConnectedComplete(this, socket.Connected);
}
No matter how many times I try, the connection is rejected ... the addresses match everywhere and yet it does not work, so I have nothing to write about in this case, especially since UDP works for me.
What I wrote only works on the same NAT network. Unfortunately, I noticed that on the same NAT created two connections. One is the result of trying to connect the new socket A to B and the other is the result of receiving a new connection from B to A, so each client has one unnecessary socket connected by a local address. So all NAT TCP / IP Punch NAT doesn't work for me. I can actually use UDP but I really need TCP. I have been sitting on it for several months in my free time but nowhere can I find an example from the code and not the theory of which there is a lot.
I accumulated a lot of knowledge for 8 years and from 2 I write applications using sockets and finally I need to punch the net.
Why won't I use the ready solution? I need my own which is fully open using only UDP and TCP because some target devices only support these protocols. I also used the Socket class but this one did not give me a working copy.
Maybe someone will be able to help me for which I would be very grateful and certainly the post will also help others understand this.
Regards
Octavian

NAT hole punching only works with UDP, and even that is a hack.
NAT firewall implementations will start tracking a TCP stream when they see the initial SYN packet leaving the network. To capture incoming TCP streams you need to create an incoming rule on the router, there's no way around this. If the router supports UPnP, you can ask it to create that rule for you dynamically.
Since UDP doesn't have a SYN packet equivalent, routers will start tracking the stream on any outgoing packet. This is why NAT hole punching works. If both end points are behind NAT and just assume that the link will work. Both can just start sending UDP packets to each other. The routers will add the connection state, and map the incoming packets to each endpoint.

Reopen Tcp socket on the same port

I have a socket that serves a single request-response purpose.
I set it up on port XXX let it wait for a connection, read the data and reply with some data.
I would like to open a new socket on the same port. As soon as the response was sent.
That is handled externally (there is a manager that is checking the state of the thread and if it was used it disposes it and creates a new one.
The problem is that it gets blocked on
_socket = _socket.Accept();
and when a new client tries to connect it never leaves this line. (And client gets no reply).
The socket is running in
new Thread(Run);
and here is my Run method:
private void Run()
{
var ipHostInfo = Dns.Resolve(Dns.GetHostName());
var ipAddress = ipHostInfo.AddressList[0];
var localEndPoint = new IPEndPoint(ipAddress, Port);
_socket = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
try
{
_socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
_socket.Bind(localEndPoint);
_socket.Listen(100);
_socket = _socket.Accept();
var data = string.Empty;
while (true)
{
var bytes = new byte[1024];
var bytesRec = _socket.Receive(bytes);
data += Encoding.UTF8.GetString(bytes, 0, bytesRec);
if (data.IndexOf("<EOF>", StringComparison.Ordinal) <= -1) continue;
var dataWithoutEof = data.Substring(0, data.IndexOf("<EOF>", StringComparison.Ordinal));
//TODO: do smt with the data
break;
}
var byteData = Encoding.UTF8.GetBytes("testResponse" + "<EOF>");
_socket.Send(byteData);
_socket.Shutdown(SocketShutdown.Both);
_socket.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
I suppose that I am not closing the existing socket correctly.

You code is wrong you should not expect this method to exit because you want your server up and running the whole time. I am assuming here you call run several times. Don't do that.
The code becomes then something like this :
_socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
_socket.Bind(localEndPoint);
_socket.Listen(100);
while (true)
{
var _servicingsocket = _socket.Accept();
....
_servicingsocket.close();
}
accept is a blocking call. That waits for a new connection.
_socket is a listening socket and must be kept during the lifetime of the server.
A TCP connection is based on the notion of a socket pair.
When the server starts you have a single socket that listens on port 100.
Suppose a connection is established, then accept returns what is called a servicing socket that is basically a clone from the listening socket. This means that it is also using source port 100, but because it is a servicing socket it belongs to a socket pair that identifies the connection. A socket pair is the combination of 2 sockets, your own socket and the peer. When a data comes in, TCP will iterate through the socket pairs to find the right socket.
An additional advantage of doing it this way is that you allow other connection attempts to queue up on the listening socket while you are processing the first request. Your _socket is overwritten with the servicing socket and you are then assuming that the listening socket is going to be garbage collected. I am not sure if this is going to happen because I haven't tried it like you are doing it in your code because it is a bad idea in the first place because it implements idisposable. (https://msdn.microsoft.com/en-us/library/system.net.sockets.socket%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396) If you really want to close the server you have to make sure to close both the servicing socket and the listening socket to make the code clean.

TcpConnection capable of handling concurrent requests. Client-Server

I'm trying to build a simple Client-Server Application with the following codes:
//SERVER
IPAddress ipAd = IPAddress.Parse("192.163.10.101");
TcpListener myList = new TcpListener(ipAd, 8001);
myList.Start();
Console.WriteLine("The server is running at port 8001...");
Console.WriteLine("The local End point is :" + myList.LocalEndpoint);
Console.WriteLine("Waiting for a connection.....");
Socket s = myList.AcceptSocket();
Console.WriteLine("Connection accepted from " + s.RemoteEndPoint);
//CLIENT
TcpClient tcpclnt = new TcpClient();
Console.WriteLine("Connecting.....");
tcpclnt.Connect("192.163.10.101",8001);
Console.WriteLine("Connected");
this actually does what I need wherein the client can connect to the server. However, when I try to run multiple instances of the client to connect with the server, the server only accepts the first client to connect. Meaning there's like a one-to-one connection wherein only one client can connect with the client. However, what I need is to give the server the ability to accept connections from more than one client.
If anyone can point me a possible solution to this, I'll really be appreciative! Thanks!

You need to call AcceptSocket again to accept another socket.
A typical design would be to have to call BeginAcceptSocket and in the callback call EndAcceptSocket, dispatch the client processing to its own thread (or a worker thread using async methods) and then call BeginAcceptSocket again.
This fragment is untested but should be more or less right/ get you thinking in the right direction.
class Server
{
public Server()
{
TcpListener listener = null;
// init the listener
listener.BeginAcceptSocket((ar) => AcceptLoop(ar, listener),null);
}
public void HandleClientSocketRead(IAsyncResult ar, byte[] recvBuffer, Socket clientSocket)
{
int recvd = clientSocket.EndReceive(ar);
//do something with the data
clientSocket.BeginReceive(recvBuffer, 0, 1024, SocketFlags.None, (ar2) => HandleClientSocketRead(ar2, recvBuffer, clientSocket), null);
}
public void AcceptLoop(IAsyncResult ar, TcpListener listener)
{
Socket clientSocket = listener.EndAcceptSocket(ar); // note that this can throw
byte[] recvBuffer = new byte[1024];
clientSocket.BeginReceive(recvBuffer, 0, 1024, SocketFlags.None, (ar2) => HandleClientSocketRead(ar2, recvBuffer, clientSocket), null);
listener.BeginAcceptSocket((ar) => AcceptLoop(ar, listener), null);
}
}

If you are looking to write a server, a good design is to have a [server].exe and a [client].exe. The [server].exe, will of course accept and process all incoming connections, maintain the client sockets, and perform whatever actions you need. Below is a very basic example on writing a server to accept multiple client sockets, and store them in a List object. This, however, is not multithreaded so the code, does block.
[server].exe
//-----------------------------------------------------------------------------
// <copyright file="Program.cs" company="DCOM Productions">
// Copyright (c) DCOM Productions. All rights reserved.
// </copyright>
//-----------------------------------------------------------------------------
namespace MultiSocketServerExample {
using System;
using System.Net.Sockets;
using System.Net;
using System.Collections.Generic;
class Program {
static List<Socket> m_ConnectedClients = new List<Socket>();
static void Main(string[] args) {
Socket host = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
host.Bind(new IPEndPoint(IPAddress.Any, 9999));
host.Listen(1);
while (true) {
m_ConnectedClients.Add(host.Accept());
Console.WriteLine("A client connected.");
}
}
}
}
Then to work with your clients: (Again, very basic example)
m_ConnectedClients[0].Send(Encoding.ASCII.GetBytes("hello!");
Network programming with the Socket class is a lot easier in my opinion then using TcpListener and TcpClient. The reason I say this is that it is already a really good and easy to use implementation, and by using TcpListener and TcpClient where they create further abstraction, lessenes your ability to understand what is going on (in my opinion).

getting 10060 (Connection Timed Out) when stress testing simple tcp server

I have created simple tcp server - it works pretty well.
the problems starts when we switch to the stress tests -since our server should handle many concurrent open sockets - we have created a stress test to check this.
unfortunately, looks like the server is choking and can not respond to new connection request in timely fashion when the number of the concurrent open sockets are around 100.
we already tried few types of server - and all produce the same behavior.
the server: can be something like the samples in this post(all produce the same behavior)
How to write a scalable Tcp/Ip based server
here is the code that we are using - when a client connects - the server will just hang in order to keep the socket alive.
enter code here
public class Server
{
private static readonly TcpListener listener = new TcpListener(IPAddress.Any, 2060);
public Server()
{
listener.Start();
Console.WriteLine("Started.");
while (true)
{
Console.WriteLine("Waiting for connection...");
var client = listener.AcceptTcpClient();
Console.WriteLine("Connected!");
// each connection has its own thread
new Thread(ServeData).Start(client);
}
}
private static void ServeData(object clientSocket)
{
Console.WriteLine("Started thread " + Thread.CurrentThread.ManagedThreadId);
var rnd = new Random();
try
{
var client = (TcpClient)clientSocket;
var stream = client.GetStream();
byte[] arr = new byte[1024];
stream.Read(arr, 0, 1024);
Thread.Sleep(int.MaxValue);
}
catch (SocketException e)
{
Console.WriteLine("Socket exception in thread {0}: {1}", Thread.CurrentThread.ManagedThreadId, e);
}
}
}
the stress test client: is a simple tcp client, that loop and open sokets, one after the other
class Program
{
static List<Socket> sockets;
static private void go(){
Socket newsock = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
IPEndPoint iep = new IPEndPoint(IPAddress.Parse("11.11.11.11"), 2060);
try
{
newsock.Connect(iep);
}
catch (SocketException ex)
{
Console.WriteLine(ex.Message );
}
lock (sockets)
{
sockets.Add(newsock);
}
}
static void Main(string[] args)
{
sockets = new List<Socket>();
//int start = 1;// Int32.Parse(Console.ReadLine());
for (int i = 1; i < 1000; i++)
{
go();
Thread.Sleep(200);
}
Console.WriteLine("press a key");
Console.ReadKey();
}
}
}
is there an easy way to explain this behavior? maybe c++ implementation if the TCP server will produce better results? maybe it is actually a client side problem?
Any comment will be welcomed !
ofer

Specify a huge listener backlog: http://msdn.microsoft.com/en-us/library/5kh8wf6s.aspx

Firstly a thread per connection design is unlikely to be especially scalable, you would do better to base your design on an asynchronous server model which uses IO Completion Ports under the hood. This, however, is unlikely to be the problem in this case as you're not really stressing the server that much.
Secondly the listen backlog is a red herring here. The listen backlog is used to provide a queue for connections that are waiting to be accepted. In this example your client uses a synchronous connect call which means that the client will never have more than 1 connect attempt outstanding at any one time. If you were using asynchronous connection attempts in the client then you would be right to look at tuning the listen backlog, perhaps.
Thirdly, given that the client code doesn't show that it sends any data, you can simply issue the read calls and remove the sleep that follows it, the read calls will block. The sleep just confuses matters.
Are you running the client and the server on the same machine?
Is this ALL the code in both client and server?
You might try and eliminate the client from the problem space by using my free TCP test client which is available here: http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html
Likewise, you could test your test client against one of my simple free servers, like this one: http://www.lenholgate.com/blog/2005/11/simple-echo-servers.html
I can't see anything obviously wrong with the code (apart from the overall design).

C#: Crash on ManualResetEvent

I wrote my code using this article at msdn as a primary helper
My code:
private ManualResetEvent _AllDone = new ManualResetEvent(false);
internal void Initialize(int port,string IP)
{
IPEndPoint _Point = new IPEndPoint(IPAddress.Parse(IP), port);
Socket _Accpt = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
_Accpt.Bind(_Point);
_Accpt.Listen(2);
while (true)
{
_AllDone.Reset();
_Accpt.BeginAccept(null, 0, new AsyncCallback(Accept), _Accpt);
_AllDone.WaitOne(); <<crash here
}
}
This is what happens,I set a breakpoint at BeginAccept(I thought there's the problem),but it steps it normally.However,when I try to step "_AllDone.WaitOne()" - the server crash.
If _allDone can't be used in a win32 form application - how do I make my project?
EDIT
I forgot to mention I wrote _AllDone.Reset() in Accept(),but It doesn't go there,I set a breakpoint there,but it won't go.
private void Accept(IAsyncResult async)
{
_AllDone.Set();
Socket _Accpt = (Socket)async.AsyncState;
Socket _Handler = _Accpt.EndAccept(async);
StateObject _State = new StateObject();
_State.workSocket = _Handler;
_Handler.BeginReceive(_State.buffer, 0, StateObject.BufferSize, 0, new AsyncCallback(ReadCallback), _State);
}

So if I get it right, you want to re-start Accept as soon as a socket connection is received, and not wait until Accept is done, and that's why you don't use the sync version of Accept.
So you are saying that it does not fire your Accept method when you connect a socket to the specified address and port? Because that's what Accept does: it accepts a new incoming connection, waiting until a client connects. So that may be why you are thinking that it "crashed" and why it never reaches your code in your Accept method.
Hint: maybe also have a look at Socket.AcceptAsync
Edit: To set up an async server listening to incoming connections, you don't need any ManualWaitEvent:
internal void Initialize(int port,string IP) {
IPEndPoint _Point = new IPEndPoint(IPAddress.Parse(IP), port);
Socket _Accpt = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
_Accpt.Bind(_Point);
_Accpt.Listen(2);
_Accpt.BeginAccept(null, 0, new AsyncCallback(Accept), _Accpt);
}
private void Accept(IAsyncResult async) {
Socket _Accpt = (Socket)async.AsyncState;
Socket _Handler;
try {
_Handler = _Accpt.EndAccept(async);
} finally {
_Accpt.BeginAccept(null, 0, new AsyncCallback(Accept), _Accpt);
}
StateObject _State = new StateObject();
_State.workSocket = _Handler;
_Handler.BeginReceive(_State.buffer, 0, StateObject.BufferSize, 0, new AsyncCallback(ReadCallback), _State);
}
Note: You will also need an exit condition, so that the BeginAccept is not called (for instance when you want to shut down the server).

I think that Lucero is trying to say, that the application works normally, you may ask how come.
Well when you are using a server side socket application, what you basically do is to ask the server to liseten to a port and wait for a conection to arrive. when the connection arrives then you do the rest of the code.
What Lucero was saying is, that while no message is arriving to the server the server keeps lisetning and waiting, which might look for you as if it's freezes.
Is it the case in your code?