I'm trying to do a project in which I have a virtual environment with cameras and want to send screenshots of selected cameras through a socket to a python client that will be running on a raspberry pi zero. In the raspberry I will have additional processing but that is not my current issue.
I've looked into screencapturing in unity, there was a project that appeared to do the capturing part quite right although I'm having some trouble adapting the code from https://assetstore.unity.com/packages/tools/camera/screenshot-helper-102472
I have
ScreenshortHelper.iCaptureWithCamera(camera, (texture2D) => {//Code}};
Would I be able to insert the "send" code in the "//Code" block?
Also that part for me is a bit tricky, sending the "texture" through the socket, I was thinking about sending some parameters like size and dimensions first so that I know what I would be receiving on the python end. Does this make sense?
I appreciate any help that you can give me!
Thanks in advance.
First, you need to configure the Transport Layer API in your unity code, and put references to your hostId and your channelId somewhere where that function can access it.
Then, that anonymous function there can be:
(texture2D) => {
// Encode it to PNG bytes and put it into a buffer:
byte[] buffer = texture2D.EncodeToPNG();
// Connect to the python listener:
byte error;
connectionId = NetworkTransport.Connect(hostId, "192.168.1.42", 8888, 0, out error);
// Send:
int bufferLength = bytes.Length;
NetworkTransport.Send(hostId, connectionId, channelId, buffer, bufferLength, out error);
// Disconnect:
NetworkTransport.Disconnect(hostId, connectionId, out error);
}
And then on the other side, you can use Python to listen on a binary stream, and then do whatever you need to do with it. There is some useful information on how to read an image over a socket in python here: Receive Image using socket programming in Python
Related
I'm developing an UWP application which has to receive a video stream from a remote pc.
Right now I'm capturing the video from the PC's webcam, sending it to a remote server which return it to me over a TCP socket.
I've been able to succesfully do this thing with an audio stream.
The problem occours when I receive a portion of the video stream as a byte array and try to create a SoftwareBitmap whach has to be represented in a XAML Image element.
The source code is structured to fire an event when a videoframe has been captured, then convert it to a byte[], write it on the TCP socket; when a message is received on the socket another event is fired in order to feed the UI with a single image.
Here the portion of the code in which i get the Exception:
var reader = (DataReader)sender;
try
{
SoftwareBitmap img = new SoftwareBitmap(BitmapPixelFormat.Bgra8, 1280, 720);
img.CopyFromBuffer(reader.ReadBuffer(reader.UnconsumedBufferLength));
ImageReadyEvent(SoftwareBitmap.Convert(img,
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Ignore), null);
}
catch (Exception ex)
{
throw;
}
The exception is fired when img.CopyFromBuffer(reader.ReadBuffer(reader.UnconsumedBufferLength)); is called.
At that moment the value of reader.UnconsumedBufferLength is 55000 byte.
The same code works perfectly if I execute it right after the video frame is ready, without sending it over the socket.
I've also tryed a BitmapDecoder but it fails everytime, with both the possible overrides of BitmapDecoder.CreateAsync();
I'm not figuring out how to solve this issue, anyone has an advice to make this thing work?
Your code is correct, It may mismatch in the buffer when transfer over TCP socket. Please try to compare the received data with the source data. And optimize your transport protocol.
I have a simple TCP server written in C#, pinging out data at a specified interval and receiving the data I have a NodeJS app.
I'm current sending the data from a NetworkStream like so:
private NetworkStream nwStream;
nwStream = clientMachine.GetStream();
nwStream.Write(buffer, 0, buffer.Length); // Inside a loop
And receiving it in NodeJS like:
client.on('data', (data) => {
Is there any way to emit a different event, other than data from my C# TCP server? Using Socket.io I can socket.emit('customEvent', someData) and receive it like socket.on('customEvent', receivedData) but I'm unsure if I can achieve this in C#.
I'm not working with Socket.io, the above is just an example of the behaviour I'd like to achieve.
If you're using raw sockets at both ends, you'll just have to design your own protocol that embeds a name into each logical frame... TCP sockets themselves don't have any such mechanism, nor do raw packets.
A typical / simple way of doing this might be something like:
{Event name, UTF8}\r\n{length, 4 bytes little endian integer}{payload, length bytes}
If your event name might include newline/linefeed, then you'd need to use the payload length/payload approach for the event name, too.
Socket.IO has it's own network protocol specified here. The encodeAsString(obj) function in 'socket.io-parser' may be what you're looking for.
Hi I have a socket programming with C# Socket Server and C++ Socket Client.
Server I use Socket.Send(bytes[]) to send message
CPP Client side i use recv(socket,buffer,length,flags)
But in server if i send multiple messages concurrently on the client side i recieve all messages as concatenated.
for(int i=0;i<10;i++)
{
var bytes= GetBytes("msg"+i);
theSocket.Send(bytes);
}
C++ Socket CLient:
Thread.Start()
{
var msg = recv(theSocketClient,buffer,1024,0);
ProcessMessageFromSocket(msg);
}
Expected is:
process msg1 then
Process msg2... Process msg10
Actual:
processMessage(msg1msg2msg3...msg10 );
What i am missing?
my attempts to fix this:
1. C# NetworkStream.Flush() -- Even after this it gives me concatenated strings
2. CPP Everytime after finishing the read i erase the buffer (but the actual socket has data concatenated so this did not help)
It's normal for it to work this way. It's up to you to define your own protocol for splitting up data, for example you might to choose to start each transmission with 4 bytes telling you how long that transmission is.
Simples - TCP cannot transfer any messages longer than one byte - it's an octet/byte stream.
If you want to transfer messages longer than one byte, you need another protocol on top of TCP.
I'm currently writing some async UDP network code in C#. I'm sending small packets (less than 50 bytes of data in each so far) back and forth and my first thought was to split them into two different packets and still send it as one packet but receive it as two. So a header or an extra information packet is always added to the start of the real packet. That would contain an ID and the data length.
So I thought I could split it on the receiving end (async receive) and first receive the header and then the actual information. This is so that I don't have to worry about the order between packets and "packet headers".
So I wrote code that basically worked like this:
Client sends 30 bytes of data to the server, where the first 3 bytes is the packet header.
The server would have called (PACKET_HEADER_SIZE = 3):
socket.BeginReceiveFrom(state.Buffer, 0, PACKET_HEADER_SIZE, SocketFlags.None, ref endPoint, ReceivePacketInfo, state);
Then receives the data:
private void ReceivePacketInfo(IAsyncResult ar)
{
StateObj state = (StateObj) ar.AsyncState;
int bytesRead = socket.EndReceiveFrom(ar, ref endPoint);
state.BytesReceived += read;
if (state.BytesReceived < state.Buffer.Length)
{
_socket.BeginReceiveFrom(state.Buffer, state.BytesReceived, state.Buffer.Length - state.BytesReceived, SocketFlags.None, ref endPoint, ReceivePacketInfo, state);
}
else
{
//my thought was to receive the rest of the packet here
}
}
but when calling socket.EndReceiveFrom(ar) I get a SocketException:
"A message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram into was smaller than the datagram itself"
So now I have a couple of questions.
Do I have to make sure I receive the whole packet (in this case both the header and the packet) before I call EndReceiveFrom?
Can I assume that I will either get the whole packet in one go or get nothing so that my if-statement in ReceivePacketInfo would be redundant (as long as it's size is less than the maximum packet size, of course)?
If I cannot, is there a good way of solving my problem? I could tag all my packet headers and all my packets to be able to map them together I suppose. I could also try to have a standardized "packet ending" so that I just read until I hit the end of the packet.
Thanks in advance for any help!
Can I assume that I will either get the whole packet in one go or get nothing
That's almost the only thing, that UDP can guarantee - the content of a packet. If packet is received it is guaranteed to have same size and same content. So you have to make sure, that you buffer is large enough for a packet.
The order of a packets is not guaranteed and the delivery itself. It is up to you and your application to handle dropped packets and out of order packets.
My application in c# wants to cominicate with 3rd party Tcp server to send data and recieve back response messages ...The syntax of commands has UShort,ULONG,BYTE type datas
a sample command that needed to send by my app is
USHORT 0xFFFF
USHORT 0x00D0
BYTE 0xDD
then in app i send data as
TcpClient tcpClient = new TcpClient();
tcpClient.Connect("XX.XX.XX.XX",portnumber);
Networkstream ns=tcpClient.GetStream();
StreamWriter sw=new StreamWriter(ns);
sw.Write(0xFFFF);
sw.Write(0x00DD);
sw.Write(0x00);
//or send them bytes
sw.Write(0xFF);
sw.Write(0xFF);
sw.Write(0x00);
sw.Write(0xD0);
sw.Write(0x00);
sw.Write(0x00);
and I read incoming messages over server as
while (true)
{
byte[] buff=new byte[tcpClient.ReceiveBufferSize];
ns.Read(buff, 0, tcpClient.ReceiveBufferSize);
string dv= BitConverter.ToString(buff));
}
//returned data looks like FF-A2-00-23-00-02-00-00-00-00-00-00-D9-2E-20-2E-00-A0-04-00-AE-08
//yes i know this byte syntaxes but returning data is not that i look response for command that i sent..
but returning values are not that i look for
Is there any wrong on my code with sending data to server??
and any recomendations on reading writing datas are welcome...
Nobody can tell you what's wrong with the response when they don't know the protocol employed. The server's sending that because it feels like it... it might be something wrong with your request, or it might be a message indicating that it's offline for service. You can only check it's specification on how to interpret the result it did send, or ask the people who maintain it.
Might also be a good idea to tag this question with the language you're using, so people can make sense of the function calls and whether you're invoking them properly.
I'd also recommend using a packet sniffer (or on Linux simply strace) to show the packets being read and written... you will probably see the mistakes there. Then, use another program to interact with the server that does work, and compare bytes.