Develop Reference

Best way to stream low latency video from a Raspberry Pi to an UWP-APP

For a project, I have to communicate with a Raspberry Pi Zero from a UWP-APP via TCP. Because both, the Raspberry and the computer with the interface, have got a private IP, I have to use a server to forward messages from one client to the other one. This part already works but now my problem is that I have to implement video streaming from the Raspberry to the UWP-APP.
Because my partner is in charge of creating and designing the UWP-APP, I have made myself a little Test-Interface with WindowsForms. I have tried several techniques like Netcat the video output over the server to the client or direct TCP-streaming with raspivid, but the best solution so far is the one I found in this project here. But instead of using the Eneter.Messaging-library I use my own class for communication with TcpClients.
I use mono to run my C# script on the Raspberry and the code to stream the Video looks like this:
while (true)
{
//Wait with streaming until the Interface is connected
while (!RemoteDeviceConnected || VideoStreamPaused)
{
Thread.Sleep(500);
}
//Check if Raspivid-Process is already running
if(!Array.Exists(Process.GetProcesses(), p => p.ProcessName.Contains("raspivid")))
raspivid.Start();
Thread.Sleep(2000);
VideoData = new byte[VideoDataLength];
try
{
while (await raspivid.StandardOutput.BaseStream.ReadAsync(VideoData, 0, VideoDataLength) != -1 && !VideoChannelToken.IsCancellationRequested && RemoteDeviceConnected && !VideoStreamPaused)
{
// Send captured data to connected clients.
VideoConnection.SendByteArray(VideoData, VideoDataLength);
}
raspivid.Kill();
Console.WriteLine("Raspivid killed");
}
catch(ObjectDisposedException)
{
}
}
Basically, this method just reads the h264 data from the Standard-Output-Stream of the raspivid process in chunks and sends it to the server.
The next method runs on the server and just forwards the byte array to the connected interface-client.
while (RCVVideo[id].Connected)
{
await RCVVideo[id].stream.ReadAsync(VideoData, 0, VideoDataLength);
if (IFVideo[id] != null && IFVideo[id].Connected == true)
{
IFVideo[id].SendByteArray(VideoData, VideoDataLength);
}
}
SendByteArray() uses the NetworkStream.Write() Method.
On the interface, I write the received byte[] to a named pipe, to which the VLC-Control connects to:
while (VideoConnection.Connected)
{
await VideoConnection.stream.ReadAsync(VideoData, 0, VideoDataLength);
if(VideoPipe.IsConnected)
{
VideoPipe.Write(VideoData, 0, VideoDataLength);
}
}
Following code initializes the pipe-server:
// Open pipe that will be read by VLC.
VideoPipe = new NamedPipeServerStream(#"\raspipipe",
PipeDirection.Out, 1,
PipeTransmissionMode.Byte,
PipeOptions.WriteThrough, 0, 10000);
And for VLC:
LibVLC libVLC = new LibVLC();
videoView1.MediaPlayer = new MediaPlayer(libVLC);
videoView1.MediaPlayer.Play(new Media(libVLC, #"stream/h264://\\\.\pipe\raspipipe", FromType.FromLocation));
videoView1.MediaPlayer.EnableHardwareDecoding = true;
videoView1.MediaPlayer.FileCaching = 0;
videoView1.MediaPlayer.NetworkCaching = 300;
This works fine on the Windowsforms-App and I can get the delay down to 2 or 3 seconds (It should be better in the end but it is acceptable). But on the UWP-App I can't get it to work even after adding /LOCAL/ to the pipe name. It shows that the VLC-Control connects to the pipe, and I can see that data is written to the pipe but it doesn't display video.
So my question is:
How can I get this to work with the VLC-Control (LibVLCSharp) in UWP? Am I missing something fundamental?
Or is there even a better way to stream the video in this case?
I have researched a bit on the UWP-MediaPlayerElement to but I can't find a way to get my byte[] into it.

First of all, thank you for your quick responses and interesting ideas!
I took a look into Desktop Bridge but it is not really what I wanted, because my colleague has already put in a lot of effort to design the UWP-APP and my Windows-Form is just a botch to try things out.
But the thing that really worked for me was StreamMediaInput . I have no idea how I missed this before. This way I just passed my NetworkStream directly to the MediaPlayer without using a Named-Pipe.
LibVLC libVLC = new LibVLC();
videoView1.MediaPlayer = new MediaPlayer(libVLC);
Media streamMedia = new Media(libVLC, new StreamMediaInput(Client.Channels.VideoConnection.stream), ":demux=h264");
videoView1.MediaPlayer.EnableHardwareDecoding = true;
videoView1.MediaPlayer.FileCaching = 0;
videoView1.MediaPlayer.NetworkCaching = 500;
videoView1.MediaPlayer.Play(streamMedia);
This solution is now working for me both, in UWP and in Windows-Forms.

How to write data to USB HID device in Android with only a single input endpoint

I have a USB HID device that I would like to communicate with. I am successfully doing so on Windows using the HidSharp library (link: https://github.com/treehopper-electronics/HIDSharp). My Windows application is developed using the .NET Framework 4.5, C#, and Visual Studio.
I now want to communicate with this same USB HID device from an Android tablet instead of from the Windows desktop. I am encountering some problems doing so. When I have the device plugged in to my tablet, it reports a single interface with a single "read" endpoint. Here is what is reported to me:
Interface #0
Class: Human Interaction Device (0x3)
Endpoint: #0
Address : 0x81 (10000001)
Number : 1
Direction : Inbound (0x80)
Type : Intrrupt (0x3)
Poll Interval : 1
Max Packet Size: 64
Attributes : 000000011
As you can see, it only reports a single endpoint, which is an inbound endpoint. I need to be able to output simple commands to this device, which I was able to do so successfully on Windows using HidSharp.
HidSharp abstracted everything into a single "stream" object that you could read from and write to. Using the Android APIs, there isn't a single "stream" object, but rather there seem to be 3 different ways of reading/writing: bulk transfer, control transfer, and USB Request. I've tried sending out data using all 3, but with seemingly no success.
Any suggestions on what to do? Is there a reason why I could send out data to this device on Windows, but seemingly cannot do so from Android? Is there a way to use a single endpoint as both a read and a write endpoint? Is there something that I am just obviously missing and not understanding?
I am using Xamarin as my development environment (C#, Visual Studio 2017). Since code is always helpful, here is how I am connecting to the device:
int VendorID = 0x04d8;
int ProductID = 0x2742;
UsbManager USB_Manager = null;
UsbDevice USB_Device = null;
UsbDeviceConnection DeviceConnection = null;
UsbInterface DeviceInterface = null;
UsbEndpoint OutputEndpoint = null;
UsbEndpoint InputEndpoint = null;
//Grab the Android USB manager and get a list of connected devices
var USB_Manager = MyMainActivity.ApplicationContext.GetSystemService(Android.Content.Context.UsbService) as Android.Hardware.Usb.UsbManager;
var attached_devices = USB_Manager.DeviceList;
//Find the device in the list of connected devices
foreach (var d in attached_devices.Keys)
{
if (attached_devices[d].VendorId == VendorID && attached_devices[d].ProductId == ProductID)
{
USB_Device = attached_devices[d];
break;
}
}
//Assuming we found the correct device, let's set everything up
if (USB_Device != null)
{
for (int j = 0; j < USB_Device.InterfaceCount; j++)
{
DeviceInterface = USB_Device.GetInterface(j);
for (int i = 0; i < DeviceInterface.EndpointCount; i++)
{
var temp_ep = DeviceInterface.GetEndpoint(i);
if (temp_ep.Type == Android.Hardware.Usb.UsbAddressing.XferInterrupt)
{
if (temp_ep.Direction == Android.Hardware.Usb.UsbAddressing.In)
{
InputEndpoint = temp_ep;
}
if (temp_ep.Direction == Android.Hardware.Usb.UsbAddressing.Out)
{
OutputEndpoint = temp_ep;
}
}
}
}
//Request permission to communicate with this USB device
UsbReceiver receiver = new UsbReceiver();
PendingIntent pending_intent = PendingIntent.GetBroadcast(Game.Activity, 0, new Android.Content.Intent(UsbReceiver.ACTION_USB_PERMISSION), 0);
IntentFilter intent_filter = new IntentFilter(UsbReceiver.ACTION_USB_PERMISSION);
Game.Activity.RegisterReceiver(receiver, intent_filter);
USB_Manager.RequestPermission(USB_Device, pending_intent);
bool has_permission = USB_Manager.HasPermission(USB_Device);
var device_connection = USB_Manager.OpenDevice(USB_Device);
device_connection.ClaimInterface(DeviceInterface, true);
DeviceConnection = device_connection;
}
Next, here is how I attempt to read from the device:
//3 methods of attempting to read from the device
//Method 1:
byte[] inpt = new byte[64];
var request = new UsbRequest();
request.Initialize(DeviceConnection, InputEndpoint);
var byte_buffer = ByteBuffer.Allocate(64);
request.Queue(byte_buffer, 64);
DeviceConnection.RequestWait();
byte_buffer.Rewind();
for(int i = 0; i < 64; i++)
{
inpt[i] = (byte) byte_buffer.Get();
}
//Method 2:
byte[] inpt = new byte[64];
DeviceConnection.BulkTransfer(InputEndpoint, inpt, inpt.Length, 1000);
//Method 3:
byte[] inpt = new byte[64];
DeviceConnection.ControlTransfer(UsbAddressing.In, 0, 0, 0, inpt, 64, 1000);
And finally, here is how I attempt to write data to this device:
//Method 1:
byte[] output_msg; //This variable is assigned elsewhere in the code
DeviceConnection.BulkTransfer(OutputEndpoint, output_msg, output_msg.Length, 30);
//Method 2:
byte[] output_msg; //This variable is assigned elsewhere in the code
DeviceConnection.ControlTransfer(UsbAddressing.Out, 0, 0, 0, output_msg, output_msg.Length, 1000);
//Method 3:
byte[] output_msg; //This variable is assigned elsewhere in the code
var write_request = new UsbRequest();
write_request.Initialize(DeviceConnection, OutputEndpoint);
var byte_buffer_write = ByteBuffer.Wrap(output_msg);
request.Queue(byte_buffer_write, output_msg.Length);
DeviceConnection.RequestWait();
"OutputEndpoint" is typically null because there is no output endpoint, so I often replace "OutputEndpoint" with "InputEndpoint", but with no success.
Any help would be greatly appreciated! Thanks!!!

You are dealing with HID devices which means you should do Interrupt Transfers.
In Android, you should use UsbRequest to perform Interrupt Transfers (as it does Asynchronous NonBlocking IO).
The endpoints are unidirectional and can be used for both inbounds and outbound (but not at the same time)
If the endpoint is inbound then submit the Urb using UsbRequest and queue as you tried before but using empty buffer with expected bufferLength.
The RequestWait will return UsbRequest Object back upon completion.
If the usbRequest.getEndPoint().getDirection() is inbound then your buffer variable will be updated with read buffer from the device.
If the usbRequest.getEndpoint().getDirection() is outbound then you should pass your buffer to write data to the device

Bluetooth BLE Battery Level - C#

I have an specific issue about the Bluetooth BLE API on Windows 10.
At the moment I'm programming a tool on C# (Visual Studio) which connects itself to a given BLE - Device. Currently the connection works perfect and I can read out the ServiceUUIDs and the CharacterUUIDs.
The main Problem is after I try to read the value of the Character its always returns me 00. I heard that implementing a notification will change that and I followed these instructions but they didn't help me.
(Here are the Specifications from bluetooth.com:)
My Code:
//connect to BluetoothDevice
var device = await BluetoothLEDevice.FromIdAsync(address);
//get UUID of Services
var services = await device.GetGattServicesAsync();
if (services != null)
{
foreach (var servicesID in services.Services)
{
//if there is a service thats same like the Battery Service
if (servicesID.Uuid.ToString() == BluetoothBLE.Constants.BATTERY_SERVICE)
{
//updateServiceList is like a console logging in my tool
updateServiceList($"Service: {servicesID.Uuid}");
var characteristics = await servicesID.GetCharacteristicsAsync();
foreach (var character in characteristics.Characteristics)
{
if (Constants.BATTERY_LEVEL == character.Uuid.ToString())
{
updateServiceList("C - UUID: "+ character.Uuid.ToString());
GattReadResult result = await character.ReadValueAsync();
if (result.Status == GattCommunicationStatus.Success)
{
var reader = DataReader.FromBuffer(result.Value);
byte[] input = new byte[reader.UnconsumedBufferLength];
reader.ReadBytes(input);
System.Diagnostics.Debug.WriteLine(BitConverter.ToString(input));
}
}
}
}
After Running my Code, the system logs 00. The characterUUID for battery level (0x2A19, from https://www.bluetooth.com/specifications/gatt/viewer?attributeXmlFile=org.bluetooth.characteristic.battery_level.xml) was read successfully but the value is strange..

Unable to read from Serial Port using C# Mono (RaspberryPi)

I'm attempting to write a C# library which looks at all available USB serial ports on a Raspberry Pi so that I can enumerate, identify and communicate with a set of Arduinos connected to the Pi via a USB hub.
I am able to make this work on my windows machine (several Arduinos connected to my desktop computer) and have even been able to make it work on my Pi however, I am struggling to understand how to generalize the fix.
If I attempt to run the program by itself on the Pi, I am able to open the serial port and send data however, I cannot receive anything from the Arduinos: I get timeout exceptions. I understand that Mono's implementation of SerialPort is limited and I must use SerialPort.ReadByte() instead of Readline() and the data received events (my solution is based on code from HowToSystemIOPorts). My Serial port enumeration is using a method outlined in another stack exchange response here.
My timeout is currently set to 4 seconds, which is several orders of magnitude longer than I expect to receive the message.
After a lot of googling, I came across mention of using minicom to initialize the serial port here, which to my surprise allowed me to receive data from the Arduino. The biggest drawback is that I need to initialize the port using minicom and leave the process opening each time I boot the Pi. I also can't seem to figure out how to make this work with multiple Arduinos.
Here is what I have tried so far:
Updated the Pi firmware and software to their latest versions
Attempted to use both an Arduino MEGA 2560 R3 and Arduino UNO
Changed the owner of the tty* ports (ttyACM0 and ttyUSB0 in this case) to both my user and group
Successfully configured the port via minicom, left the process running and start the program and read/wrote data. A manual process which only seems to work for one Arduino at a time
Successfully run the program in Windows without fault
Verified the Arduinos are recognized by the Pi running "dmesg | grep tty"
Here is what I hope to solve:
Automatic setup/initialization of the Arduino serial ports. Whether through a shell script run before the main program or within Mono code so that the code below can run as intended.
Here is my connection code:
public bool StartArduinoComms()
{
string[] ports = GetPortNames();
foreach (string port in ports)
{
mLogger.LogMessage(ProsthesisCore.Utility.Logger.LoggerChannels.Arduino, string.Format("Found serial port {0}", port));
}
bool foundCorrectArduino = false;
var idPacket = new ArduinoMessageBase();
idPacket.ID = ArduinoMessageValues.kIdentifyValue;
string jsonOutput = Newtonsoft.Json.JsonConvert.SerializeObject(idPacket);
foreach (string port in ports)
{
SerialPort serialPort = new SerialPort(port, kArduinoCommsBaudRate);
serialPort.Parity = Parity.None;
serialPort.DataBits = 8;
serialPort.StopBits = StopBits.One;
//Only check unopened ports
if (!serialPort.IsOpen)
{
serialPort.Open();
//Disable telemtry just incase
var toggle = new { ID = ArduinoMessageValues.kTelemetryEnableValue, EN = false };
string disableTelem = Newtonsoft.Json.JsonConvert.SerializeObject(toggle);
serialPort.Write(disableTelem);
//Discard any built up data
serialPort.DiscardInBuffer();
serialPort.Write(jsonOutput);
serialPort.ReadTimeout = kIDTimeoutMilliseconds;
string response = string.Empty;
for (int i = 0; i < kNumRetries; ++i)
{
try
{
//This is guaranteed to timeout if not configured through minicom
response = ReadLine(serialPort);
break;
}
//Catch case where the serial port is unavailable. MOve to next port
catch (TimeoutException)
{
continue;
}
}
if (!string.IsNullOrEmpty(response))
{
//Perform response validation
}
else
{
//Got no response
}
if (!foundCorrectArduino)
{
serialPort.Close();
}
}
}
return foundCorrectArduino;
}
/// <summary>
/// From https://stackoverflow.com/questions/434494/serial-port-rs232-in-mono-for-multiple-platforms
/// </summary>
/// <returns></returns>
private static string[] GetPortNames()
{
int p = (int)Environment.OSVersion.Platform;
List<string> serial_ports = new List<string>();
// Are we on Unix?
if (p == 4 || p == 128 || p == 6)
{
string[] ttys = System.IO.Directory.GetFiles("/dev/", "tty*");
foreach (string dev in ttys)
{
//Arduino MEGAs show up as ttyACM due to their different USB<->RS232 chips
if (dev.StartsWith("/dev/ttyS") || dev.StartsWith("/dev/ttyUSB") || dev.StartsWith("/dev/ttyACM"))
{
serial_ports.Add(dev);
}
}
}
else
{
serial_ports.AddRange(SerialPort.GetPortNames());
}
return serial_ports.ToArray();
}

Have a look at stty command. It will let you set/read teminal settings
http://linux.about.com/od/lna_guide/a/gdelna38t01.htm will give a rundown on it's use.
It would be easier to call out to than minicom, and the settings stay on the device.

I have done something like the same as you before.
I had to read and write data through USB Serial adapter, and didnt use minicom.
It may not be god code but i found that inorder to read the data I could create a new thread and have that check for data, my code include a lot of stuff but basicly i did this:
System.Threading.Thread newThread;
newThread = new System.Threading.Thread(this.check_get_data);
and the check_get_data method
public void check_get_data ()
{
byte tmpByte = 0;
while (m_objSerialPort.BytesToRead != 0) {
tmpByte = (byte)m_objSerialPort.ReadByte ();
DoSomethingWithByte(tmpByte);
Thread.Sleep(20);
}
}
this is currently running with two usbserials. dont know if it helps but hope you find your solution

MonoDroid Bluetooth

I have been working with Monodroid for a few days and still can't figure out how to send a command through Bluetooth.
This is my scenario: I have a Tablet/Cellphone working with Android 2.1+ and need to send and receive data to a Bluetooth printer (in bytes).
What i managed so far:
using Android.Bluetooth; // library necessary
BluetoothAdapter bth = BluetoothAdapter.DefaultAdapter;
if (!bth.IsEnabled)
bth.Enable();
ICollection<BluetoothDevice> bthD = bth.BondedDevices;
foreach (BluetoothDevice d in bthD)
{
if (d.Name == "DPP-350")
{
Java.Util.UUID UUID = Java.Util.UUID.FromString("00001101-0000-1000-8000-00805F9B34FB");
// Get the BLuetoothDevice object
BluetoothSocket s = d.CreateRfcommSocketToServiceRecord(UUID);
s.Connect();
// Try to send command
...
s.Close()
}
}
The program asks for the pairing info, with is done correctly.
I have tried many ways to send the command:
// the command
// Self_Test = Chr(27) + Chr(84) = ESC T
byte[] dBytes = System.Text.Encoding.GetEncoding(1252).GetBytes(Self_Test);
// wont work
new Java.IO.ObjectOutputStream(s.OutputStream).Write(dBytes);
// wont work
System.IO.Stream st = s.OutputStream;
if (st.CanWrite)
{
st.Write(dBytes, 0, dBytes.Length);
st.Flush();
}
// wonk work
s.OutputStream.Write(dBytes, 0, dBytes.Length);
s.OutputStream.Flush();
No error is raised. I'm running out of options here...
Thanks in advance!

I know this is a very old thread, but I wanted to post a reply so others will know the answer. I too searched hard with no luck.
s.OutputStream.BeginWrite(buffer, 0, buffer.Length,new AsyncCallback(delegate {}), State.Connected);
Thanks.