Develop Reference

How to communicate between two Unity apps with TCP?

Update
I figured out what the problem was. I was trying to move too much data over TCP, and it was causing freeze-ups. For some reason, this wasn't manifesting in the editor...who knows for what reason. If anyone else stumbles upon this problem (in a program like Unity, where functions are looping constantly and data is always being processed), consider that you're moving too much irrelevant data.
Original Post
I've run into quite the problem, and I'm hoping I can receive some guidance.
In short, I'm wondering how to use TCP to communicate two Unity apps over the same computer. I've gotten it functioning in editor, but when both apps are built, communication quickly breaks down.
This is really stumping me, because I don't understand why an app would work in the Editor environment, but not in the official build.
When I use TCP to communicate between two Unity apps (on the same computer), it works so long as one of them is kept in the Unity environment. That is, if I build one app, and open the other in the Unity editor, TCP communication works flawlessly.
Here is some more background: One of my apps is functioning as a User Interface, and the other is interfacing with a Looking Glass to provide a holographic display of in-game objects. Originally, they were combined into one App - but I had a lot of trouble getting Unity's multidisplay support to function between two monitors of different resolutions. Looking Glass factory even provides a prefab to do just this, but it is broken in the current SDK. So I have resorted to using sockets to interface between two apps, one for each monitor.
I'm using C#'s TCP listener class: https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.tcplistener?view=netframework-4.8
And TCP client class: https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.tcpclient?view=netframework-4.8
Presently, the UI is acting as the TCPListener, and the application that produces holograms is the TCPClient. Within each of these applications, I'm using two Queues - an IncomingMessages queue and an Outgoing Messages queue - which are global variables shared between the main thread and the networking thread.
TCP Listener:
private void Start()
{
incomingMessages = new Queue();
outgoingMessages = new Queue();
Application.runInBackground = true;
thread = new Thread(new ThreadStart(Receive));
thread.Start();
//stuff happens that's irrelevant to this question. And then...
}
void Receive()
{
TcpListener server = null;
try
{
// Set the TcpListener on port 13000.
Int32 port = 13000;
IPAddress localAddr = IPAddress.Parse("127.0.0.1");
// TcpListener server = new TcpListener(port);
server = new TcpListener(localAddr, port);
// Start listening for client requests.
server.Start();
// Buffer for reading data
Byte[] bytes = new Byte[256];
String data = null;
// Enter the listening loop.
Debug.Log("About to reenter main while in Server...");
while (threadContinue)
{
Debug.Log("Waiting for a connection... ");
// Perform a blocking call to accept requests.
// You could also user server.AcceptSocket() here.
TcpClient client = server.AcceptTcpClient();
Debug.Log("Connected!");
data = null;
// Get a stream object for reading and writing
NetworkStream stream = client.GetStream();
int i;
// Loop to receive all the data sent by the client.
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
// Translate data bytes to a ASCII string.
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
Debug.Log("Received from Client: " + data);
lock (this)
incomingMessages.Enqueue(data);
string response = supplyData();
byte[] msg = System.Text.Encoding.ASCII.GetBytes(response);
// Send back a response.
stream.Write(msg, 0, msg.Length);
Debug.Log("Sent to Client: " + response);
}
// Shutdown and end connection
client.Close();
}
}
catch (SocketException e)
{
Debug.Log("SocketException: ");
Debug.Log(e);
}
finally
{
// Stop listening for new clients.
server.Stop();
}
Debug.Log("Exiting 'Receive'");
}
And here is the TCP Client. It attempts to connect a regular intervals, and also whenever new data is available. This is so that it can receive information from the server regularly and share new data whenever it is available:
void Start()
{
//prepare networking
Application.runInBackground = true;
outgoingMessages = new Queue();
incomingMessages = new Queue();
thread = new Thread(new ThreadStart(Connect));
thread.Start();
//stuff happens that's irrelevant to this question...
}
private void Connect()
{
String server = "127.0.0.1";
Int32 port = 13000;
string message = "";
while (threadContinue == true)
{
if (timeToConnect())
{
lastConnection = ourTime;
if (outgoingMessages.Count > 0)
message = outgoingMessages.Dequeue().ToString();
else
message = "Nothing to report.";
try
{
// Create a TcpClient.
// Note, for this client to work you need to have a TcpServer
// connected to the same address as specified by the server, port
// combination.
client = new TcpClient(server, port);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Debug.Log("Sent to Server: " + message);
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
lock (this)
incomingMessages.Enqueue(responseData);
Debug.Log("Received from Server: " + responseData);
stream.Close();
client.Close();
}
catch (ArgumentNullException e)
{
Debug.Log("ArgumentNullException: ");
Debug.Log(e);
outgoingMessages.Enqueue(message);
}
catch (SocketException e)
{
Debug.Log("SocketException: ");
Debug.Log(e);
outgoingMessages.Enqueue(message);
}
}
}
}
private bool timeToConnect()
{
if ((ourTime - lastConnection > NETWORK_DELAY) || (outgoingMessages.Count > 0))
return true;
return false;
}
Instantiated in separate threads so that Unity's main thread can continue unhindered.
Again - it works in Editor, but when I build it, it breaks.

Update
I figured out what the problem was. I was trying to move too much data over TCP, and it was causing freeze-ups. For some reason, this wasn't manifesting in the editor...just in the exported app. Who knows for what reason. If anyone else stumbles upon this problem...where you're bypassing Unity's multidisplay functionality by building multiple apps that communicate over network...consider that you're burdening your queues with too much data.

Is a named pipe able to do what i want to do?

This is take II, i posted the other week and my question was put on hold, i adjusted my text but could not get a review, and the system closed the original post.
Logic
Server Side: Read only - Server Opens pipe then at regular interval checks if there is content (i.e. not at end of stream) and reads info. This check has to be poll based as only during the poll is there a valid context to pass on the data..
Client Side: Write-only - Open pipe, write to pipe, close (client.exe called many times, has short life span, code below is test code), e.g. some other script will "call client.exe with info"
Can this work flow be handled in pipes ? e.g. snippet of client code shown only the first client message is seen by the "server"
If pipes can do this after coding tips as most examples are for client - servers having similar life cycles.
Code snippets
for (int i = 0; i < 10; i++)
{
//Client - simulate exe starting and ending
var client = new NamedPipeClientStream(".", "PipesOfPiece", PipeDirection.Out, PipeOptions.WriteThrough);
client.Connect();
StreamWriter writer = new StreamWriter(client);
Console.WriteLine("Client about to send message");
writer.WriteLine("Called from client i = {0}", i);
writer.Close();
client.Close();
Thread.Sleep(5000);
}
// server snippet
var server = new NamedPipeServerStream("PipesOfPiece", PipeDirection.In);
server.WaitForConnection(); <= can this we optional with code below
StreamReader reader = new StreamReader(server);
while (true)
{
// simulate start of poll code
if (server.IsConnected)
{
if (!reader.EndOfStream)
{
var line = reader.ReadToEnd();
Console.WriteLine("Server: {0}", line);
}
} // End of poll code
Thread.Sleep(1000);
}
// server snippet
var server = new NamedPipeServerStream("PipesOfPiece", PipeDirection.In);
server.WaitForConnection(); <= can this we optional with code below
StreamReader reader = new StreamReader(server);
while (true)
{
// simulate start of poll code
if (server.IsConnected)
{
if (!reader.EndOfStream)
{
var line = reader.ReadToEnd();
Console.WriteLine("Server: {0}", line);
}
} // End of poll code
Thread.Sleep(1000);
}
So I am rusty on my pipes, I am hoping that a pipe can be opened, written to then read, and the waitforconnect() is there for the cases where you want this and is optional. I suppose it all triggers around who owns the pipe, i.e. if the server opens a pipe and is waiting for someone to write for it, why does it need to wait for a connect ? (I am hoping the server is the owner so when it ends, the pipe disappears)

Without a good, minimal, complete code example that reliably reproduces whatever specific problem you are having, it is impossible to provide specific advice as to how to fix that problem. However, I can at least try to answer some of your questions about how named pipes can be used, and provide a code example to illustrate some of the concepts.
First, some rules and observations:
A pipe instance can be used for only one connection. Note that pipes inherit Stream, and streams have a very specific paradigm: you open one, read to the end, and then you're done with the stream. Some streams, like FileStream, are seekable but even there you are only ever dealing with a single resource (i.e. the original file…you can't reconnect a FileStream to a different file), and network streams aren't even seekable.
A pipe must be connected before you perform I/O on it.
You may have multiple instances of pipes with the same name (if you initialize them correctly…by default, you may only have one pipe of any given name).
Clients trying to connect to a named pipe will wait until such a pipe exists. It does not need to exist at the time the client initiates the connection.
Only one client can connect to any given instance of a pipe. Any given instance of a server pipe can only ever handle a single client during its entire lifetime (see the very first point above).
So, what about your questions?
Can this work flow be handled in pipes ?
If I understand the work-flow correctly, yes. But you need to be careful to implement it correctly.
As I understand it, you want for your server to only attempt to read from clients periodically. At the same time, you want for a client to be able to write to a pipe at any time. This can be done, but it won't be straightforward.
Note per the above that you cannot open a single server pipe, and then let multiple clients periodically connect and disconnect from that pipe. Once the first client has connected, the pipe is no longer usable. It's a stream, and that first client's disconnection causes the stream to reach its end. It's done.
Note also that while a client can attempt to connect to a pipe that doesn't exist yet, it will wait until it can. So if you want your clients to not have to wait until the polling interval has expired, you'll need to maintain a server pipe available to connect to at all times.
But you've already said that you won't be able to handle data read from the server pipe at arbitrary points in time, but rather only during your polling interval.
Because pipes don't inherently support this specific scenario, IMHO the right way to implement it is to separate the behaviors into two different components. Maintain a simple pipe server that opens a pipe, waits for a client to connect, reads whatever the client has sent, closes the pipe, and then starts over.
Then have an intermediary class that can act as the go-between for the server I/O and whatever component you have that ultimately receives the data. This intermediary will retain a copy of the data after it's been received (the pipe code will deliver it to the intermediary as soon as it's received, regardless of the polling interval); the data will then later be retrieved by the polling component on its next polling interval (i.e. when the "context" as you put it is in fact available to which to deliver the data).
I am hoping that a pipe can be opened, written to then read, and the waitforconnect() is there for the cases where you want this and is optional
Unfortunately, your hope doesn't match the reality. Pipes can be bidirectional; i.e. "written to then read". But WaitForConnect() is not optional. The server must wait for a connection before attempting to read from the pipe, and for that pipe instance it will only ever be able to receive data from a single client.
I am hoping the server is the owner so when it ends, the pipe disappears
The server process is the one that actually creates the pipe. So yes, in that sense it is the owner. And yes, when the server process is terminated, any pipes it's created are destroyed.
Below, please find a simple code example that illustrates the use of multiple and concurrent servers and clients. You can adjust the numbers of each with the declared constants at the top of the example.
When running it, note that if more clients are active than servers, the additional clients will simply wait until a server pipe is available to connect to. Once one is, they will connect and proceed normally. If there are at least as many server pipe instances as there are clients trying to connect, all of the clients are serviced concurrently.
// NOTE: as a sample program, contrary to normal and correct
// programming practices error-handling has been omitted, and
// non-awaited async methods have been declared as void.
class Program
{
private const string _kserverName = "TestSO33093954NamedPipeClients";
private const int _kmaxServerCount = 3;
private const int _kmaxClientCount = 3;
static void Main(string[] args)
{
StartServers(_kmaxServerCount);
StartClients(_kmaxClientCount);
Console.WriteLine("Clients are being started. Press return to exit program.");
Console.ReadLine();
}
private static async void StartClients(int clientCount)
{
for (int i = 0; i < clientCount; i++)
{
RunClient(i);
await Task.Delay(300);
}
}
private static async void RunClient(int instance)
{
NamedPipeClientStream client = new NamedPipeClientStream(
".", _kserverName, PipeDirection.InOut, PipeOptions.Asynchronous);
client.Connect();
ReadClient(client);
using (StreamWriter writer = new StreamWriter(client))
{
writer.AutoFlush = true;
for (int i = 0; i < 5; i++)
{
string text =
string.Format("Instance #{0}, iteration #{1}", instance, i);
Console.WriteLine("Client send: " + text);
await writer.WriteLineAsync(text);
await Task.Delay(1000);
}
client.WaitForPipeDrain();
}
}
private static async void ReadClient(Stream stream)
{
using (TextReader reader = new StreamReader(stream))
{
string line;
while ((line = await reader.ReadLineAsync()) != null)
{
Console.WriteLine("Client recv: " + line);
}
}
}
private static void StartServers(int maxServerInstances)
{
for (int i = 0; i < maxServerInstances; i++)
{
RunServer(maxServerInstances);
}
}
private static async void RunServer(int maxServerInstances)
{
while (true)
{
using (NamedPipeServerStream server = new NamedPipeServerStream(
_kserverName, PipeDirection.InOut, maxServerInstances,
PipeTransmissionMode.Byte, PipeOptions.Asynchronous))
{
await server.WaitForConnectionAsync();
byte[] buffer = new byte[1024];
int bytesRead;
Decoder decoder = Encoding.UTF8.GetDecoder();
while ((bytesRead =
await server.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
int cch = decoder.GetCharCount(buffer, 0, bytesRead);
char[] rgch = new char[cch];
decoder.GetChars(buffer, 0, bytesRead, rgch, 0);
Console.Write("Server recv: " + new string(rgch));
await server.WriteAsync(buffer, 0, bytesRead);
}
}
}
}
}
static class PipeExtensions
{
// As I am not running with .NET 4.6 yet, I need this little helper extension
// to wrap the APM-based asynchronous connection-waiting with the await-friendly
// Task-based syntax. Anyone using .NET 4.6 will have this in the framework already
public static Task WaitForConnectionAsync(this NamedPipeServerStream server)
{
return Task.Factory.FromAsync(
server.BeginWaitForConnection, server.EndWaitForConnection, null);
}
}

c# service to listen to a port

I believe what I am looking to create is a service that listens to a specific port, and when data is sent to that port, it sends off that data to another script for processing.
For some reason though, the service times out when I try to start it. My logs tells me TcpClient client = server.AcceptTcpClient(); is where it is stopping (actually, it is getting stuck on 'starting' in Services).
Since I have no experience with C#, making services, or working with servers in this manner, the code is pretty much just what I found online.
The OnStart method looks like this.
protected override void OnStart(string[] args)
{
try
{
TcpListener server = null;
// Set the TcpListener on port 13000.
Int32 port = 1234;
IPAddress localAddr = IPAddress.Parse("127.0.0.1");
// TcpListener server = new TcpListener(port);
server = new TcpListener(localAddr, port);
// Start listening for client requests.
server.Start();
// Buffer for reading data
Byte[] bytes = new Byte[256];
String data = null;
// Enter the listening loop.
while (true)
{
// Perform a blocking call to accept requests.
// You could also user server.AcceptSocket() here.
TcpClient client = server.AcceptTcpClient();
data = null;
// Get a stream object for reading and writing
NetworkStream stream = client.GetStream();
int i;
// Loop to receive all the data sent by the client.
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
// Translate data bytes to a ASCII string.
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
// Process the data sent by the client.
data = data.ToUpper();
byte[] msg = System.Text.Encoding.ASCII.GetBytes(data);
// Send back a response.
stream.Write(msg, 0, msg.Length);
}
// Shutdown and end connection
client.Close();
}
}
catch (SocketException e)
{
}
finally
{
}
}

As per MSDN, TcpServer.AcceptTcpClient blocks, so you're probably never returning from your Service's OnStart method, which causes the service to never actually "start".
You might consider using another thread and return from OnStart as soon as possible.
Cheers

As far as creating the Windows service itself, you should be able to use this link, even though it's dated. This companion link shows how to have the service install and uninstall itself. Finally, use this link to understand how to have your service run constantly and how to properly respond to start and stop commands.
To have your service interact with the socket, you'll want to modify the WorkerThreadFunc() from the last link. This is where you should start listening for and processing inbound socket connections.

make server-client work unlimitedly

How can I make the server and the client to run unlimitedly and be able to exchange data(meaning, until the application is closed), instead of run for one exchange of information only.
Tried with while(true) but maybe didn't put it on the right place and then I can't really reach the methods for closing and stopping the socket and the listener.
Here's some of the code of the server:
public static void StartServer()
{
try
{
IPAddress ip = IPAddress.Parse("192.168.1.11");
TcpListener myListener = new TcpListener(ip, 8000);
myListener.Start();
Socket s = myListener.AcceptSocket();
byte[] b = new byte[100];
int k = s.Receive(b);
... some other actions ...
s.Close();
myListener.Stop();
}
and then then Main() where I invoke it.
With the Client is the same story.

You can create an infinite loop which contains the Receive function processing data, and returns to receive. That way the server always excepts data from the client until server, or client terminates.
while(true)
{
byte[] buffer = new byte[100];
s.Receive(buffer);
//Do something with data...
}
Beware through because in your current design only one client is supported. If you want to support multiple clients consider using threads.

getting 10060 (Connection Timed Out) when stress testing simple tcp server

I have created simple tcp server - it works pretty well.
the problems starts when we switch to the stress tests -since our server should handle many concurrent open sockets - we have created a stress test to check this.
unfortunately, looks like the server is choking and can not respond to new connection request in timely fashion when the number of the concurrent open sockets are around 100.
we already tried few types of server - and all produce the same behavior.
the server: can be something like the samples in this post(all produce the same behavior)
How to write a scalable Tcp/Ip based server
here is the code that we are using - when a client connects - the server will just hang in order to keep the socket alive.
enter code here
public class Server
{
private static readonly TcpListener listener = new TcpListener(IPAddress.Any, 2060);
public Server()
{
listener.Start();
Console.WriteLine("Started.");
while (true)
{
Console.WriteLine("Waiting for connection...");
var client = listener.AcceptTcpClient();
Console.WriteLine("Connected!");
// each connection has its own thread
new Thread(ServeData).Start(client);
}
}
private static void ServeData(object clientSocket)
{
Console.WriteLine("Started thread " + Thread.CurrentThread.ManagedThreadId);
var rnd = new Random();
try
{
var client = (TcpClient)clientSocket;
var stream = client.GetStream();
byte[] arr = new byte[1024];
stream.Read(arr, 0, 1024);
Thread.Sleep(int.MaxValue);
}
catch (SocketException e)
{
Console.WriteLine("Socket exception in thread {0}: {1}", Thread.CurrentThread.ManagedThreadId, e);
}
}
}
the stress test client: is a simple tcp client, that loop and open sokets, one after the other
class Program
{
static List<Socket> sockets;
static private void go(){
Socket newsock = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
IPEndPoint iep = new IPEndPoint(IPAddress.Parse("11.11.11.11"), 2060);
try
{
newsock.Connect(iep);
}
catch (SocketException ex)
{
Console.WriteLine(ex.Message );
}
lock (sockets)
{
sockets.Add(newsock);
}
}
static void Main(string[] args)
{
sockets = new List<Socket>();
//int start = 1;// Int32.Parse(Console.ReadLine());
for (int i = 1; i < 1000; i++)
{
go();
Thread.Sleep(200);
}
Console.WriteLine("press a key");
Console.ReadKey();
}
}
}
is there an easy way to explain this behavior? maybe c++ implementation if the TCP server will produce better results? maybe it is actually a client side problem?
Any comment will be welcomed !
ofer

Specify a huge listener backlog: http://msdn.microsoft.com/en-us/library/5kh8wf6s.aspx

Firstly a thread per connection design is unlikely to be especially scalable, you would do better to base your design on an asynchronous server model which uses IO Completion Ports under the hood. This, however, is unlikely to be the problem in this case as you're not really stressing the server that much.
Secondly the listen backlog is a red herring here. The listen backlog is used to provide a queue for connections that are waiting to be accepted. In this example your client uses a synchronous connect call which means that the client will never have more than 1 connect attempt outstanding at any one time. If you were using asynchronous connection attempts in the client then you would be right to look at tuning the listen backlog, perhaps.
Thirdly, given that the client code doesn't show that it sends any data, you can simply issue the read calls and remove the sleep that follows it, the read calls will block. The sleep just confuses matters.
Are you running the client and the server on the same machine?
Is this ALL the code in both client and server?
You might try and eliminate the client from the problem space by using my free TCP test client which is available here: http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html
Likewise, you could test your test client against one of my simple free servers, like this one: http://www.lenholgate.com/blog/2005/11/simple-echo-servers.html
I can't see anything obviously wrong with the code (apart from the overall design).