I'm trying to send some data to a Python server through a ZMQ Socket. The Python server creates a socket using ZMQ as well.
```lang-python
print("Connecting with Unity Toolkit...")
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind('tcp://*:5555')
while True:
request = socket.recv_multipart()
print("Somethings received");
req = json.loads(request[1])
print("Received from Unity Toolkit: ", req)
```
I've tried to communicate with that server using a Python ZMQ client and it works, but when I try to do it with the .NET ZMQ library inside of an Unity 3D project, there is an exception that I don't know how to handle. The NETQMClient I was given (this is legacy code from an older project, but I'm allowed to change it) does something like this:
```lang-csharp
//this lines are executed using a Thread()
AsyncIO.ForceDotNet.Force();
requestSocket = new RequestSocket();
requestSocket.Connect("tcp://192.162.0.104:5555");
isAvailable = true;
while (!clientStopped){
//Debug.Log("Continuing");
}
requestSocket.Close();
NetMQConfig.Cleanup();
//end of the function running on a thread
//...
//later, when a button is pressed
try {
//endpoint is a one-character string that
//the server uses as a cue for something
requestSocket.SendMoreFrame(endpoint);
requestSocket.SendFrame(request); //this a class turned into a JSON using JsonUtility.ToJson
} catch (System.Exception) {
Debug.Log("Something went wrong");
throw;
}
```
Later the C# script waits for a response and I handle that response and use it. The thing is that I've tested it running the client and the server in the same computer and it works. HOWEVER, when I deploy the project into the real Hololens 2 (where the NETQM client runs), I get this error in Visual Studio.
I added the try catch block to see if there was some strange behaviour but there is no Exception in that point. Any hints on the right directions are appreciated.
EDIT: Whole Scripts used in this project for replicating the bug.
This class holds the data that will later be converted to JSON
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public static class Serialization {
public int numPanels;
public bool colorHarmony;
public float colorfulness;
[Serializable]
public Request(int numPanels, bool colorHarmony, float colorfulness) {
this.numPanels = numPanels;;
this.colorHarmony = colorHarmony;
this.colorfulness = colorfulness;
}
}
This is the script with the ZMQ logic
using System.Collections.Generic;
using System.Threading;
using NetMQ;
using NetMQ.Sockets;
using UnityEngine;
public class PythonNetworking {
private bool clientStopped;
private RequestSocket requestSocket;
private byte[] frame;
// for now only one request at a time is supported
public string requestResult;
private bool isAvailable;
public PythonNetworking() {
clientStopped = false;
var clientThread = new Thread(NetMQClient);
clientThread.Start();
}
public void StopClient() {
clientStopped = true;
}
// ReSharper disable once InconsistentNaming
private void NetMQClient() {
AsyncIO.ForceDotNet.Force();
requestSocket = new RequestSocket();
// Computer running the server which must receive the info
requestSocket.Connect("tcp://192.168.0.104:5555");
isAvailable = true;
while (!clientStopped)
{
//Debug.Log("Continuing");
}
requestSocket.Close();
NetMQConfig.Cleanup();
}
public void SetFrame(byte[] currFrame) {
frame = currFrame;
}
// Create queue of requests in case multiple have to be handled
private void SimpleRequest(string endpoint, string request) {
// wait until socket is available
while (!isAvailable) {
//Debug.Log("Socket unavailable");
}
isAvailable = false;
if (request == null) {
requestSocket.SendFrame(endpoint);
} else {
Debug.Log("Sending to Python server: " + request);
try
{
requestSocket.SendMoreFrame(endpoint);
Debug.Log("SendMoreFrame called");
requestSocket.SendFrame(request);
Debug.Log("SendFrame called");
}
catch (System.Exception)
{
Debug.Log("Something went wrong");
throw;
}
}
var msg = requestSocket.ReceiveFrameBytes();
Debug.Log("ReceiveFrameBytes returned a value");
isAvailable = true;
requestResult = System.Text.Encoding.UTF8.GetString(msg);
}
public void PerformRequest(string endpoint, string request) {
requestResult = null;
var requestThread = new Thread(() => SimpleRequest(endpoint, request));
requestThread.Start();
}
}
This is the script making the request when an event is triggered:
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using Microsoft.MixedReality.Toolkit.Input;
using Microsoft.MixedReality.Toolkit.UI;
using Newtonsoft.Json;
public class MyTool: MonoBehaviour {
private PythonNetworking pythonNetworking;
public struct PanelConstraints {
public string name;
public float height;
public float width;
}
public void Start() {
pythonNetworking = new PythonNetworking();
Debug.Log("Tool initialized!");
//called here for debugging purposes, this method is called when a button is pressed
SubmitConstraints();
}
//When a button is pressed
public void SubmitConstraints() {
Debug.Log("Submitting constraints...");
StartCoroutine(CreateRequest("P"));
}
private IEnumerator CreateRequest(string type) {
Serialization.Request request = new Serialization.Request(1, false, 1.0f);
var requestJson = JsonUtility.ToJson(request);
pythonNetworking.PerformRequest(type, requestJson);
yield return new WaitUntil(() => pythonNetworking.requestResult != null);
if (type == "P") {
panelData = JsonConvert.DeserializeObject<List<string>>(pythonNetworking.requestResult);
}
}
}
Finally, the Python server code is:
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind('tcp://*:5555')
while True:
request = socket.recv_multipart()
print("Somethings received");
req = json.loads(request[1])
#Data is processed
As per the setup, I'm using Unity 2019.4.26f1, I create a 3D scene, I add a GameObject to it, I attach the script MyTool.cs to it and I attach a button under the previous GameObject. On clicking the button, the SubmitContrainst() method is called (now called automatically in Start() for debugging purposes). This scene is deployed in the Hololens 2 using Visual Studio 2019 (although I've tried to run it in the Game Mode, communicating with a different computer to recreate the deployment environment, that is, with the client and the server in different devices and with different implementations of the library). When the application is about to start in the Hololens, the exception I introduced previously (see screenshot) is thrown. The Python server, running with Python version 3.10.4, doesn't receive anything. TCP communication seems to work, since when I run two Python scripts in different hosts performing client/server communication with ZMQ (using REQ and REP), it works, but I get the feeling this communication doesn't work on the Hololens 2. But this feels very strange because I've managed to communicate the Hololens 2 with another device (using another library, Windows.Networking.Socket).
Would you navigate to [Unity] -> [Player Settings] -> [Publishing Settings] to check the Capability of your project and make sure Internet Client or Internet Client Server is enabled according to the purpose. It can also be reviewed and modified in the manifest file of C# project.
The code around the thrown exception is more like Unity generated, you need to add break point to see if this issue is caused by a failed socket connection.
In addtion, you may also refer to the discussion on Web Socket Client Disconnecting Exception Being Thrown in UWP Builds - Unity Forum.
Related
I have a DEALER/DEALER connection.
For some reason, there seems to be an importance to who runs first. If I run the C# one (NetMQ), followed by the python one, the message sent via C# will block until the python dealer is running (as expected - Send blocks until it can send). However...
sometimes, the response I send from the python dealer is not received by the C# poller (the method added to ReceiveReady isn't called). This seems to happen when I send a message from C#, then wait a few (tried ~10) seconds before I run the python client.
However the moment I either dispose of the poller on the C# end or the socket on the python end, I get the message (in the C# app, I seem to only get the first message. But if I dispose of the socket on the python end, I get all messages at once).
I'm not sure what's going on. I would have guessed some kind of timeout on the messages, if it weren't for the fact they end up being received.
C# code (about half the lines are just for disposing):
using UnityEngine;
using AsyncIO;
using NetMQ;
using NetMQ.Sockets;
using System;
using System.Threading.Tasks;
public class MsgClient_BASIC : MonoBehaviour
{
DealerSocket socket;
NetMQPoller poller;
private bool requesterIsStarted;
public static MsgClient_BASIC this_msgclient;
private void OnReceive(object s, NetMQ.NetMQSocketEventArgs a)
{
bool more;
string FS = a.Socket.ReceiveFrameString(out more);
Debug.Log(FS);
}
void Start()
{
this_msgclient = this;
ForceDotNet.Force();
socket = new DealerSocket();
socket.Bind("tcp://localhost:5558");
poller = new NetMQPoller { socket };
socket.ReceiveReady += OnReceive;
poller.RunAsync();
}
private void Update()
{
if (Input.GetKeyDown("m"))
{
Debug.Log("Sending S");
socket.SendFrame("S");
}
}
void OnDestroy()
{
Debug.Log("Stopping async poller");
try
{
poller.StopAsync();
}
catch
{
}
Debug.Log("Disposing Poller");
try
{
poller.Dispose();
}
catch
{
}
Debug.Log("Closing socket");
try
{
socket.Close();
}
catch
{
}
Debug.Log("Disposing of socket");
try
{
((IDisposable)socket).Dispose();
}
catch
{
}
Debug.Log("Cleaning up");
NetMQConfig.Cleanup(true);
}
}
Python code:
import zmq
import json
from datetime import datetime as dt
from os import path
context = zmq.Context()
class Client:
def __init__(self,port):
self.socket = context.socket(zmq.DEALER)
self.socket.setsockopt(zmq.LINGER,500)
self.socket.connect("tcp://localhost:%d" % port)
def check_msgs(self):
try:
response = self.socket.recv_string(flags=zmq.NOBLOCK)
except zmq.Again as e:
response = None
print(response)
return response
def send_dict(self,d):
jd = json.dumps(d)
self.socket.send_string(jd,flags=zmq.NOBLOCK)
def destroy(self):
self.socket.close()
context.destroy(5)
print("Destroyed")
I have an Azure IOThub that contains one edge device. Within this edge device I have several modules running and I can change the twin property of any individual module by connecting to it with it's connection string.
Now I would like the module to do something when it's twin property is changed but the module doesn't have access to it's connection string and it shouldn't because it shouldn't need to connect to itself.
How can a module detect it's twin property change without having a connection string?
I have followed this tutorial but this uses a connection string to detect changes: https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-csharp-csharp-module-twin-getstarted#create-a-module-identity
Here's the code for this module:
using System;
using Microsoft.Azure.Devices.Client;
using Microsoft.Azure.Devices.Shared;
using System.Threading.Tasks;
using Newtonsoft.Json;
namespace DataSyncService
{
class Program
{
private const string ModuleConnectionString = "CONNECTION STRING";
private static ModuleClient Client = null;
static void ConnectionStatusChangeHandler(ConnectionStatus status,
ConnectionStatusChangeReason reason)
{
Console.WriteLine("Connection Status Changed to {0}; the reason is {1}",
status, reason);
}
static void Main(string[] args)
{
Microsoft.Azure.Devices.Client.TransportType transport =
Microsoft.Azure.Devices.Client.TransportType.Amqp;
try
{
Client =
ModuleClient.CreateFromConnectionString(ModuleConnectionString, transport);
Client.SetConnectionStatusChangesHandler(ConnectionStatusChangeHandler);
// I want to set this callback but this requires a client and the client requires
// a connection string.
Client.SetDesiredPropertyUpdateCallbackAsync(OnDesiredPropertyChanged, null).Wait();
Console.WriteLine("Retrieving twin");
var twinTask = Client.GetTwinAsync();
twinTask.Wait();
var twin = twinTask.Result;
Console.WriteLine(JsonConvert.SerializeObject(twin.Properties));
Console.WriteLine("Sending app start time as reported property");
TwinCollection reportedProperties = new TwinCollection();
reportedProperties["DateTimeLastAppLaunch"] = DateTime.Now;
Client.UpdateReportedPropertiesAsync(reportedProperties);
}
catch (AggregateException ex)
{
Console.WriteLine("Error in sample: {0}", ex);
}
Console.WriteLine("Waiting for Events. Press enter to exit...");
Console.ReadLine();
Client.CloseAsync().Wait();
}
private static async Task OnDesiredPropertyChanged(TwinCollection desiredProperties,
object userContext)
{
Console.WriteLine("desired property change:");
Console.WriteLine(JsonConvert.SerializeObject(desiredProperties));
Console.WriteLine("Sending current time as reported property");
TwinCollection reportedProperties = new TwinCollection
{
["DateTimeLastDesiredPropertyChangeReceived"] = DateTime.Now
};
await Client.UpdateReportedPropertiesAsync(reportedProperties).ConfigureAwait(false);
}
}
}
When you create a new module in Visual Studio Code, it generates a template module for you that will show you how it's done. I'll include the important bit below:
static async Task Init()
{
MqttTransportSettings mqttSetting = new MqttTransportSettings(TransportType.Mqtt_Tcp_Only);
ITransportSettings[] settings = { mqttSetting };
// Open a connection to the Edge runtime
ModuleClient ioTHubModuleClient = await ModuleClient.CreateFromEnvironmentAsync(settings);
await ioTHubModuleClient.OpenAsync();
Console.WriteLine("IoT Hub module client initialized.");
// Register callback to be called when a message is received by the module
await ioTHubModuleClient.SetInputMessageHandlerAsync("input1", PipeMessage, ioTHubModuleClient);
}
The way this works is because the Azure IoT Edge runtime will create the module as a Docker container with the connection settings as environment variables. The module client uses those variables to connect to IoT Hub when you call
ModuleClient ioTHubModuleClient = await
ModuleClient.CreateFromEnvironmentAsync(settings);
There is a good sample on this Microsoft tutorial that also covers listening for Twin updates.
I'm developing a Xamarin android app where at some point (after a successful transaction) the app must send an information email to the customer. The whole process of creating the MimeMessage instance as well as the actual sending of it is done through a task. The message (which most of the time contains linked images) is created flawlessly but as soon as I call a method to actually send it the execution freezes at that call and the whole task remains active but frozen at that point.
Here's my SendMail method which, beside of the MimeMessage parameter, receives a MailAccount class with information about the smtp machine, port and sender account. As soon as a call is made to the Connect method of the MailKit's SmtpClient instance I'm having the "frozen" behavior I described. I tried the whole thing many times - not even once the code continued beyond that line.
private static int SendMail(MailAccount account, MimeMessage message)
{
using (var client = new SmtpClient())
{
try
{
client.Connect(account.Server, account.Port, false);
}
catch (SmtpCommandException)
{
return -1;
}
catch (SmtpProtocolException)
{
return -2;
}
client.AuthenticationMechanisms.Remove("XOAUTH2");
if (client.Capabilities.HasFlag(SmtpCapabilities.Authentication))
{
try
{
client.Authenticate(account.User, Decrypt(account.Pass));
}
catch (AuthenticationException)
{
return -3;
}
catch (SmtpCommandException)
{
return -1;
}
catch (SmtpProtocolException)
{
return -2;
}
}
int status = 1; //assume success
try
{
client.Send(message);
}
catch (SmtpCommandException ex)
{
switch (ex.ErrorCode)
{
case SmtpErrorCode.MessageNotAccepted:
status = -6;
break;
case SmtpErrorCode.SenderNotAccepted:
status = -5;
break;
case SmtpErrorCode.RecipientNotAccepted:
status = -4;
break;
default:
status = -7;
break;
}
}
client.Disconnect(true);
return status;
}
}
In general, a hang while calling client.Connect suggests that the host or port that you are trying to connect with is not available. Check that the host and port are correct.
If they are correct, make sure that you are using the latest version of MailKit as some older versions used Socket.BeginConnect (string, int ...) which, on IPv6-enabled OS's, would sometimes try to use IPv4 over IPv6 and hang in some situations (such as when a virus scanner like Avast was used).
Newer versions of MailKit (>= 2.1.4 and pre-2.0 versions) do a DNS lookup of the remote host and manually specify the correct address family to use when instantiating the Socket (IPv4 vs IPv6) so as to avoid this problem.
It's also possible that the hang is caused by SSL/TLS certificate revocation checks with an unresponsive Certificate Authority. You can disable certificate revocation checks using the following code snippet:
client.CheckCertificateRevocation = false;
Just make sure to set that property before making the call to Connect.
It may be worth testing access to your SMTP server by creating a new Console application in Visual Studio and pasting in the following code and checking that it works:
using System;
using System.Threading;
using System.Threading.Tasks;
using MailKit;
using MailKit.Net.Smtp;
namespace ConsoleApp {
public class Program
{
static void Main (string[] args)
{
using (var client = new SmtpClient (new ProtocolLogger ("smtp.log"))) {
using (var cts = new CancellationTokenSource (60000)) {
try {
client.Connect ("your-host.com", 25, false, cts.Token);
client.Disconnect (true);
} catch (Exception ex) {
Console.WriteLine ("Error connecting: {0}", ex.Message);
Console.WriteLine (ex.StackTrace);
}
}
}
}
}
}
It would also be interesting to know if your Android app has problems connecting to smtp.gmail.com on port 465 (useSsl = true).
I am working on HoloLens (Unity-UWP) and trying to make a connection with PC (UWP) or Android phone work (Xamarin). So far I tried client and host with both Bluetooth and TCP (even two versions with different libraries) on Android and UWP. I kept the code entirely separated from user interface, so that it is easier to use, to understand and modular. An Action<string> is used to output results (error logs and sent messages).
Everything that is not on the HoloLens works fine (even though it's exactly the same code). It worked from PC (UWP) to Android with client and host switched. But it doesn't even work between HoloLens and PC (UWP). The behavior ranged from crashes (mostly for Bluetooth) to instant disconnection. The last tests resulted in disconnection once bytes are about to be received. It could even read the first 4 bytes (uint for the length of the following UTF-8 message), but then it was disconnected. The other devices seemed to work fine.
What I know: Capabilities are set, the code works, the issue is likely something that is common for everything that has to do with networking and HoloLens.
So the question is, is Unity or HoloLens incompatible with something I am using? What I used which is worth mentioning: StreamSocket, BinaryWriter, BinaryReader, Task (async, await). Or is HoloLens actively blocking communication with applications on other devices? I know it can connect to devices with Bluetooth and that it can connect via TCP, and it looks like people succeed to get it to work. Are there known issues? Or is there something with Unity that causes this - a bad setting maybe? Do I have to use async methods or only sync? Are there incompatibility issues with Tasks/Threads and Unity? Is this possibly the issue (inability to consent to permissions)?
Another thing to note is that I cannot ping HoloLens via its IP by using the cmd, even though the IP is correct.
I'd appreciate any advice, answer or guess. I can provide more information if requested (see also the comments below). I would suggest to focus on the TCP connection as it seemed to be working better and appears to be more "basic." Here is the code:
using System;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using Windows.Networking;
using Windows.Networking.Sockets;
#region Common
public abstract class TcpCore
{
protected StreamSocket Socket;
protected BinaryWriter BWriter;
protected BinaryReader BReader;
protected Task ReadingTask;
public bool DetailedInfos { get; set; } = false;
public bool Listening { get; protected set; }
public ActionSingle<string> MessageOutput { get; protected set; } = new ActionSingle<string> (); // Used for message and debug output. They wrap an Action and allow safer use.
public ActionSingle<string> LogOutput { get; protected set; } = new ActionSingle<string> ();
protected const string USED_PORT = "1337";
protected readonly Encoding USED_ENCODING = Encoding.UTF8;
public abstract void Disconnect ();
protected void StartCommunication ()
{
Stream streamOut = Socket.OutputStream.AsStreamForWrite ();
Stream streamIn = Socket.InputStream.AsStreamForRead ();
BWriter = new BinaryWriter (streamOut); //{ AutoFlush = true };
BReader = new BinaryReader (streamIn);
LogOutput.Trigger ("Connection established.");
ReadingTask = new Task (() => StartReading ());
ReadingTask.Start ();
}
public void SendMessage (string message)
{
// There's no need to send a zero length message.
if (string.IsNullOrEmpty (message)) return;
// Make sure that the connection is still up and there is a message to send.
if (Socket == null || BWriter == null) { LogOutput.Trigger ("Cannot send message: No clients connected."); return; }
uint length = (uint) message.Length;
byte[] countBuffer = BitConverter.GetBytes (length);
byte[] buffer = USED_ENCODING.GetBytes (message);
if (DetailedInfos) LogOutput.Trigger ("Sending: " + message);
BWriter.Write (countBuffer);
BWriter.Write (buffer);
BWriter.Flush ();
}
protected void StartReading ()
{
if (DetailedInfos) LogOutput.Trigger ("Starting to listen for input.");
Listening = true;
while (Listening)
{
try
{
if (DetailedInfos) LogOutput.Trigger ("Starting a listen iteration.");
// Based on the protocol we've defined, the first uint is the size of the message. [UInt (4)] + [Message (1*n)] - The UInt describes the length of the message (=n).
uint length = BReader.ReadUInt32 ();
if (DetailedInfos) LogOutput.Trigger ("ReadLength: " + length.ToString ());
MessageOutput.Trigger ("A");
byte[] messageBuffer = BReader.ReadBytes ((int) length);
MessageOutput.Trigger ("B");
string message = USED_ENCODING.GetString (messageBuffer);
MessageOutput.Trigger ("Received Message: " + message);
}
catch (Exception e)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus (e.HResult) == SocketErrorStatus.Unknown)
{
// Seems to occur on disconnects. Let's not throw().
Listening = false;
Disconnect ();
LogOutput.Trigger ("Unknown error occurred: " + e.Message);
break;
}
else
{
Listening = false;
Disconnect ();
break;
}
}
}
LogOutput.Trigger ("Stopped to listen for input.");
}
}
#endregion
#region Client
public class GTcpClient : TcpCore
{
public async void Connect (string target, string port = USED_PORT) // Target is IP address.
{
try
{
Socket = new StreamSocket ();
HostName serverHost = new HostName (target);
await Socket.ConnectAsync (serverHost, port);
LogOutput.Trigger ("Connection successful to: " + target + ":" + port);
StartCommunication ();
}
catch (Exception e)
{
LogOutput.Trigger ("Connection error: " + e.Message);
}
}
public override void Disconnect ()
{
Listening = false;
if (BWriter != null) { BWriter.Dispose (); BWriter.Dispose (); BWriter = null; }
if (BReader != null) { BReader.Dispose (); BReader.Dispose (); BReader = null; }
if (Socket != null) { Socket.Dispose (); Socket = null; }
if (ReadingTask != null) { ReadingTask = null; }
}
}
#endregion
#region Server
public class GTcpServer : TcpCore
{
private StreamSocketListener socketListener;
public bool AutoResponse { get; set; } = false;
public async void StartServer ()
{
try
{
//Create a StreamSocketListener to start listening for TCP connections.
socketListener = new StreamSocketListener ();
//Hook up an event handler to call when connections are received.
socketListener.ConnectionReceived += ConnectionReceived;
//Start listening for incoming TCP connections on the specified port. You can specify any port that's not currently in use.
await socketListener.BindServiceNameAsync (USED_PORT);
}
catch (Exception e)
{
LogOutput.Trigger ("Connection error: " + e.Message);
}
}
private void ConnectionReceived (StreamSocketListener listener, StreamSocketListenerConnectionReceivedEventArgs args)
{
try
{
listener.Dispose ();
Socket = args.Socket;
if (DetailedInfos) LogOutput.Trigger ("Connection received from: " + Socket.Information.RemoteAddress + ":" + Socket.Information.RemotePort);
StartCommunication ();
}
catch (Exception e)
{
LogOutput.Trigger ("Connection Received error: " + e.Message);
}
}
public override void Disconnect ()
{
Listening = false;
if (socketListener != null) { socketListener.Dispose (); socketListener = null; }
if (BWriter != null) { BWriter.Dispose (); BWriter.Dispose (); BWriter = null; }
if (BReader != null) { BReader.Dispose (); BReader.Dispose (); BReader = null; }
if (Socket != null) { Socket.Dispose (); Socket = null; }
if (ReadingTask != null) { ReadingTask = null; }
}
}
#endregion
Coincidentially, I just implemented a BT connection between HoloLens and an UWP app. I followed the sample at https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/BluetoothRfcommChat.
As capabilities, I set "Bluetooth" (of course), "Internet (Client & Server)" and "Private Networks (Client & Server)". The steps on the server side then are:
Create an RfcommServiceProvider for your own or an existing (eg OBEX object push) service ID.
Create a StreamSocketListener and wire its ConnectionReceived Event.
Bind the service Name on the listener: listener.BindServiceNameAsync(provider.ServiceId.AsString(), SocketProtectionLevel.BluetoothEncryptionAllowNullAuthentication);
If you have a custom service ID, set its name along with other attributes you may want to configure. See the sample linked above for this. I think, this is mostly optional.
Start advertising the BT service: provider.StartAdvertising(listener, true);
Once a client connects, there is a StreamSocket in the StreamSocketListenerConnectionReceivedEventArgs that you can use to create a DataReader and DataWriter on like on any other stream. If you only want to allow one client, you can also stop advertising now.
On the client side, you would:
Show the DevicePicker and let the user select the peer device. Do not forget setting a filter like picker.Filter.SupportedDeviceSelectors.Add(BluetoothDevice.GetDeviceSelectorFromPairingState(true)); You can also allow unpaired devices, but you need to call PairAsync before you can continue in step 2. Also, I think there is no way to circumvent the user consent dialogue in this case, so I would recommend pairing before. To be honest, I did not check whether the unpaired stuff works on HoloLens.
You get a DeviceInformation instance from the picker, which you can use to obtain a BT device like await BluetoothDevice.FromIdAsync(info.Id);
Get the services from the device like device.GetRfcommServicesAsync(BluetoothCacheMode.Uncached); and select the one you are interested in. Note that I found that the built-in filtering did not behave as expected, so I just enumerated the result and compared the UUIDs manually. I believe that the UWP implementation performs a case-sensitive string comparison at some point, which might lead to the requested service not showing up although it is there.
Once you found your service, which I will call s from now on, create a StreamSocket and connect using socket.ConnectAsync(s.ConnectionHostName, s.ConnectionServiceName, SocketProtectionLevel.PlainSocket);
Again, you can not create the stream readers and writers like on the server side.
The answer is Threading.
For whoever may have similar issues, I found the solution. It was due to Unity itself, not HoloLens specifically. My issue was that I wrote my code separately in an own class instead of commingle it with UI code, which would have made it 1. unreadable and 2. not modular to use. So I tried a better coding approach (in my opinion). Everybody could download it and easily integrate it and have basic code for text messaging. While this was no issue for Xamarin and UWP, it was an issue for Unity itself (and there the Unity-UWP solution as well).
The receiving end of Bluetooth and TCP seemed to create an own thread (maybe even something else, but I didn't do it actively), which was unable to write on the main thread in Unity, which solely handles GameObjects (like the output log). Thus I got weird log outputs when I tested it on HoloLens.
I created a new TCP code which works for Unity but not the Unity-UWP solution, using TcpClient/TcpListener, in order to try another version with TCP connection. Luckily when I ran that in the editor it finally pointed on the issue itself: The main thread could not be accessed, which would have written into the UI - the text box for the log output. In order to solve that, I just had to use Unity's Update() method in order to set the text to the output. The variables themselves still could be accessed, but not the GameObjects.
So what I am specifically trying to do is get a serial proxy going to pipe an arduino into Unity3d to run on linux. System.IO.Ports just doesn't work on linux with unity.
So I have gotten a python based serial proxy script, I have that up and running just fine. I can netcat into that localhost and actually get output from the arduino.
nc localhost 8082
g'day from bitty 1.0 -- type 'logout' to disconnect
Connecting to /dev/tty.usbmodem5d11... connected.
HELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLOHELLO
I have the arduino only printing out HELLO on repeat.
But the serial proxy is up and running, and has data going over it.
Now I am trying to write the code in Unity to receive this data, and I just can't get it to work.
This is my code for that:
public Socket socket = new Socket(AddressFamily.InterNetwork,SocketType.Stream,ProtocolType.Tcp);
public byte[] buffer = new byte[256];
int pinRead = 0;
void Start ()
{
socket.Connect(IPAddress.Parse("127.0.0.1"),8082);
}
void Update ()
{
if (socket.IsBound)
{
try
{
int bytesRead;
bytesRead = socket.Receive(buffer);
string incommingdata = System.Text.Encoding.ASCII.GetString(buffer);
Debug.Log(bytesRead+" || "+incommingdata);
}
catch (System.Exception e)
{
}
}
bytesRead is ALWAYS 0 and incomingData just doesn't have anything. It connects, and isBound returns true. I just can't get the code right to receive the data and put it in a format that is usable.
Please help. I need to get this working and its far out of my area of expertise. How do I make this work?
So I got this to work using tcpclient objects and some snippets of code that were posted on a random unity blog....
Here it is if anyone wants to see. I would still be really curious to know how to get the Socket implementation functioning though.
using UnityEngine;
using System.Collections;
using System.Net;
using System.Net.Sockets;
using System;
using System.Text;
using System.IO;
using System.Threading;
public class ArduinoTest : MonoBehaviour
{
private bool socketReady = false;
private TcpClient mySocket;
private NetworkStream theStream;
private StreamWriter theWriter;
private StreamReader theReader;
private string Host = "localhost";
private Int32 Port = 9900;
private int pinRead = 0;
void Start ()
{
setupSocket();
}
void Update ()
{
string read = readSocket();
if (read != "")
{
int value = int.Parse(read);
if ( value > 53)
{
Debug.Log((value-54)+" DOWN");
}
else
{
Debug.Log(value+" UP");
}
}
}
void OnApplicationQuit()
{
writeSocket("logout");
closeSocket();
}
public void setupSocket()
{
try
{
mySocket = new TcpClient(Host, Port);
theStream = mySocket.GetStream();
theWriter = new StreamWriter(theStream);
theReader = new StreamReader(theStream);
socketReady = true;
}
catch (Exception e)
{
Debug.Log("Socket error:" + e);
}
}
public void writeSocket(string theLine)
{
if (!socketReady)
return;
String tmpString = theLine + "\r\n";
theWriter.Write(tmpString);
theWriter.Flush();
}
public string readSocket()
{
if (!socketReady)
return "";
if (theStream.DataAvailable)
return theReader.Read().ToString();
return "";
}
public void closeSocket()
{
if (!socketReady)
return;
theWriter.Close();
theReader.Close();
mySocket.Close();
socketReady = false;
}
public void maintainConnection()
{
if(!theStream.CanRead)
{
setupSocket();
}
}
}
(Sorry I can't post comments yet or would do so there)
BytesRead = 0 typically means the remote side disconnected.
For what it's worth, your code works fine when I run a test TCP server listening for connections on port 8082 and sending out some text messages. So I doubt the problem is on the C# client-side.
Beyond this simple example, "getting this to work" is not necessarily a simple question. Here are a couple things you should keep in mind:
You are making a blocking Receive call, which means you will need to call Update in a loop.
TCP is a streaming protocol, so you might get partial messages or many messages bundled together in the same Read call. Look up "TCP Framing" to get more details on this.
You can try using telnet on this socket, just to see if you can connect to it.
telnet 127.0.0.1 8082