The goal of the project is to stream video captured from a python host to a c# client via tcp sockets.
Relavent python2 server script:
import cv2
import numpy as np
import socket
from threading import Thread
_continue = True
def imageStreamer4():
global _continue
cam = cv2.VideoCapture(0)
camSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
camSocket.bind(("",8081))
camSocket.listen(1)
# set flip image to false if you don't want the image to be flipped
flipImage = True
while _continue:
try:
client,address = camSocket.accept()
print("client connected")
ret,camImage = cam.read()
if flipImage:
camImage = cv2.flip(camImage,1)
#uncomment the below code to view the webcam stream locally
"""
cv2.imshow('image',camImage)
if cv2.waitKey(1) == 27:
break # esc to quit
"""
byteString = bytes(cv2.imencode('.jpg', camImage)[1].tostring())
fileSize = len(byteString)
totalSent = 0
client.send(str(fileSize).encode())
sizeConfirmation = client.recv(1024)
totalSent = 0
while totalSent < fileSize:
totalSent += client.send(byteString[totalSent:])
print(str(fileSize), str(totalSent),sizeConfirmation.decode('utf-8'))
except Exception as e:
print(e)
print("shutting down video stream")
_continue = False
print("video stream exited.")
Relevant c# client code:
using System.Collections;
using UnityEngine;
using System;
using System.Net;
using System.Text;
using System.Net.Sockets;
using System.Threading;
using UnityEngine.UI;
using System.IO;
void getVideoStream()
{
byte[] header;
int recieved;
int fileSize;
NetworkStream dataStream;
MemoryStream ms;
while (connectCam)
{
fileSize = 0;
recieved = 0;
camClient = new TcpClient(camIP, camPort);
//get header
dataStream = camClient.GetStream();
while (!dataStream.DataAvailable)
{
//waste time
}
header = new byte[1024];
dataStream.Read(header, 0, header.Length);
fileSize = Int32.Parse(Encoding.Default.GetString(bytesReducer(header)));
byte[] result = Encoding.ASCII.GetBytes(fileSize.ToString());
//send response
dataStream.Write(result, 0, result.Length);
ms = new MemoryStream();
while (!dataStream.DataAvailable)
{
//waste time
}
while (recieved < fileSize)
{
byte[] data = new byte[camClient.ReceiveBufferSize];
recieved += dataStream.Read(data, 0, data.Length);
ms.Write(data, 0, data.Length);
}
//the below class simply sends function calls from secondary thread back to the main thread
UnityMainThreadDispatcher.Instance().Enqueue(convertBytesToTexture(ms.ToArray()));
dataStream.Close();
camClient.Close();
}
}
void convertBytesToTexture(byte[] byteArray) {
try
{
camTexture.LoadImage(byteArray); //Texture2D object
camImage.texture = camTexture; //RawImage object
}
catch (Exception e)
{
print(e);
}
}
The byte counts sent and received match as they should. I'm admittedly new to working with sockets but I'm pretty certain that the data is arriving whole and intact. Unfortunately I really have no idea why the image is splitting as it is. (As shown in the above image) If it's relevant at all, both the server and client functions are being run on their own separate threads.
I've run the scripts on separate hosts and clients and the results remain the same.If any different information is required to help, just ask. I'll be happy to update as required.
I was able to bring the picture together properly by removing empty space inserted by the following code:
while (recieved < fileSize)
{
byte[] data = new byte[camClient.ReceiveBufferSize];
recieved += dataStream.Read(data, 0, data.Length);
ms.Write(data, 0, data.Length);
}
replacing it with this:
int increment = 0;
while (recieved < fileSize)
{
byte[] data = new byte[camClient.ReceiveBufferSize];
increment = dataStream.Read(data, 0, data.Length);
recieved += increment;
ms.Write(data.Take(increment).ToArray(), 0, increment);
}
So instead of taking an array the size of the client's buffer into the memory stream(even if it wasn't full), it's being condensed to only the amount of information received via the read method. This effectively removed all the blank space in the received image.
Because Tcp is a stream you should consider it as a flow of bytes so you should apply some mechanism to identify each image
this way you can apply:
A delimiter:
use a set of bytes to separate each image( but this byte set must not repeated in your stream) for example:
1- send image bytes
2-send delimiter
in the client you start to read until reach the delimiter
Consider defining a protocol and send the image size first and then send image data for example :
1-first 8 bytes is always image size
2- then the image bytes
and when you read data on the client first you always read 8 bytes and then base on this size you read the rest of data.
Another way is to use fixed size data(which is a bit hard for you in this case) you get a fix size byte array in the server and if the image data is less than the byte array fill it with zero and on the client always read bytes as many as the server array size.
Server Send byte[2048] and Client reads until the bytes which is readed reach 2048
Related
I have the following code to send a picture to a receiving application
public static void sendFile(string file, string ip)
{
using (TcpClient client = new TcpClient())
{
client.Connect(IPAddress.Parse(ip), 44451);
//Console.WriteLine(ip);
NetworkStream nwStream = client.GetStream();
MemoryStream ms = new MemoryStream();
Image x = Image.FromFile(file);
x.Save(ms, x.RawFormat);
byte[] bytesToSend = ms.ToArray();
nwStream.Write(bytesToSend, 0, bytesToSend.Length);
nwStream.Flush();
client.Close();
}
}
and I'm receiving the file on the other end with this
NetworkStream nwStream = clientCopy.GetStream();
byte[] buffer = new byte[clientCopy.ReceiveBufferSize];
//---read incoming stream---
int bytesRead = nwStream.Read(buffer, 0, clientCopy.ReceiveBufferSize);
MemoryStream ms = new MemoryStream(buffer);
Image returnImage = Image.FromStream(ms);
//ms.Flush();
//ms.Close();
String path;
if (!Directory.Exists(path = #"C:\Users\acer\AppData\Roaming\test"))
{
Directory.CreateDirectory(#"C:\Users\acer\AppData\Roaming\test");
}
string format;
if (ImageFormat.Jpeg.Equals(returnImage.RawFormat))
{
format = ".jpg";
}
else if (ImageFormat.Png.Equals(returnImage.RawFormat))
{
format = ".png";
}
else
{
format = ".jpg";
}
returnImage.Save(#"C:\Users\acer\AppData\Roaming\test\default_pic" + format, returnImage.RawFormat);
If i'm sending a picture that is small (around <20kb) the file is received 100% on the other end but if I send a file around >=100kb, the picture is received but only half of the image is loaded. I'm aware of the approach of reading the stream until all data is read but I don't know how to implement it right.
Thank you
You're only calling Read once, which certainly isn't guaranteed to read all the bytes. You could either loop, calling Read and copying the relevant number of bytes on each iteration, or you could use Stream.CopyTo:
var imageStream = new MemoryStream();
nwStream.CopyTo(imageStream);
// Rewind so that anything reading the data will read from the start
imageStream.Position = 0;
... or you could just read the image straight from the network stream:
// No need for another stream...
Image returnImage = Image.FromStream(nwStream);
(It's possible that would fail due to the stream being non-seekable... in which case using CopyTo as above would be the simplest option.)
The TCP protocol (like any other stream protocol) can't be used to transfer data as is. Most of the time it is impossible to know whether all data is arrived or whether it is received unrelated chunk of data together with the expected one. Therefore it is almost always needed to define underlying protocol, for example by sending a message header (like in HTTP) or defining a message separator (like line break in Telnet; however, using separators for big size messages are impractical). In most simple case it is enough to define very simple header that contains only the length of the message
Thus, in your case you can send 4 byte image length and then the image. On the server side you will read the 4 bytes size and then in the loop call the Read until complete message is recieved.
Please note that you can receive more bytes than expected. It means that the last chunk contains the beginning of the next message.
I am working on a project that uses a python script to send an image taken with a webcam to a c# webserver using sockets. I am currently sending strings to the server from the python client using code like:
info = bytearray("Text to send", "UTF-8")
socket.send(info)
to send data to the c# server and it functions perfectly for text and numbers. I run into an issue when trying to encode the data read from the .bmp into "UTF-8", as trying this returns an error of not being able to encode certain characters into UTF-8.
I was wondering if anyone had any idea of a way to encode this that c# will be able to recognize, or, if there is a better way of trying to implement this process, I am all ears.
A couple of options I have come up with would be to 1 - use something like google drive to save the image to, or an FTP server and then have the c# server retrieve it from there or 2 - create a packet system containing the RGB values and recreating the image from those pixel values on the server.
Thanks for your help.
EDIT: I have tried sending the file this way
data = bytearray("123456789","UTF-8")
file = open("image.bmp", "rb")
data += file.read()
socket.send(data)
and was able to successfully retreive the string "123456789", but the data after this is garbage. I have also implemented sending the size of the file before sending the data and that size number is retrieved fine, but the img data saves as a black bmp.
Edit 2 :
Here is the server and client code I am using to try and recreate the image using a memory stream. I the client code is using the process mentioned by hcalves.
Client
if __name__ == "__main__":
sock = socket.socket()
sock.connect(("localhost", 50839))
with open("image.bmp", "rb") as fd:
buf = fd.read(1024)
while (buf):
sock.send(buf)
buf = fd.read(1024)
sock.close()
Server
Socket client = server.AcceptSocket();
NetworkStream stream = new NetworkStream(client);
byte[] imgData = new byte[1024];
MemoryStream memstream = new MemoryStream();
stream.Read(imgData,0, 1024);
int counter = 0;
while (stream.DataAvailable)
{
memstream.Write(imgData, 0, 1024);
stream.Read(imgData, 0, 1024);
counter = counter + 1024;
}
memstream.Seek(0, SeekOrigin.Begin);
using (Stream file = File.OpenWrite("img.bmp"))
{
byte[] buffer = new byte[8*1024];
int len;
while ((len = memstream.Read(buffer, 0, buffer.Length)) > 0)
{
file.Write(buffer,0,len);
}
}
You shouldn't need more than this recipe:
import socket
if __name__ == "__main__":
sock = socket.socket()
sock.connect(("localhost", 50839))
with open("data.bin", "rb") as fd:
buf = fd.read(1024)
while (buf):
sock.send(buf)
buf = fd.read(1024)
sock.close()
For practical reasons, you can treat str objects (the result of fd.read) as raw data, you don't need any further crazy encoding. Just iterate the file and send over the socket. Test by running this server which just echoes to stdout with python server.py > data2.bin:
import socket
import sys
if __name__ == "__main__":
sock = socket.socket()
sock.bind(("localhost", 50839))
sock.listen(1)
client, address = sock.accept()
buf = client.recv(1024)
while (buf):
sys.stdout.write(buf)
buf = client.recv(1024)
client.close()
sock.close()
A checksum shows the binary data is sent correctly:
% md5 data.bin data2.bin
MD5 (data.bin) = 8b3280072275badf3e53a6f7aae0b8be
MD5 (data2.bin) = 8b3280072275badf3e53a6f7aae0b8be
Your C# server should be able to accept this data as is. If it doesn't work, it's because your server is expecting something in particular, not just raw data.
i have a simplistic file server\client application ive written in c#. but i commonly run into the problem that my stream writes two different reads into a single buffer. i have a synchronized stream, still isnt helping. any suggestions? thanks!
System.Threading.Thread.Sleep(25);
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
if (bytesread == 0)
{
finished = true;
System.Threading.Thread.Sleep(25);
}
string string_1 = utf.GetString(receive_fspos).TrimEnd(new char[] { (char)0 });
int fsposition = (int)Convert.ToInt64(string_1);
bytestosend = fsposition;
filestream.Position = fsposition;
byte[] buffer_1 = new byte[bufsize];
int bytesreadfromfs = filestream.Read(buffer_1, 0, buffer_1.Length);
stream_1.Write(buffer_1, 0, buffer_1.Length);
Console.Write("\rSent " + fsposition + " / " + length + " bytes");
finished = true;
I would not recommend writing your own stream method if you do not fully understand it.
The problem that you are having is because the incoming data is a stream of bytes that does not give you a way of knowing how many bytes in length that the message is.
In the code below you are stating that you would like to read "receive_fspos.Length" bytes of the stream. Since "receive_fspos.Length" is 30, the amount of bytes that will be read will be anywhere from 0 to 30.
If there is only 15 bytes that have been received by the connection. It will give you 15 bytes. If the message was 20 bytes long. Then the message is now split up into different segments.
If the first message was 4 bytes and the second message is 12 bytes. Now you have 2 messages and a set of 16 blank bytes at the end. Even worse those 16 "blank" bytes could be the beginning of a third message coming in to the stream.
If the message is 50 bytes long. Then you will only receive half of the message. Now you would need to add the bytes that were read to a seperate buffer. Read from the stream again. Then repeat this until you have determined that you have read the exact amount of bytes that are needed to complete the entire message. Then concat all of the read bytes back to a single byte[].
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
Instead of rolling your own loop please use the BCL methods. It sounded like you are using strings so this would be the preferred method.. I would suggest the following.
using(NetworkStream networkStream = tcpClient.GetStream())
using(StreamReader streamReader = new StreamReader(networkStream))
using(StreamWriter streamWriter = new StreamWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a string
string incomingString = stream.ReadLine();
//To send a string
stream.WriteLine(messageToSend);
stream.Flush();
}
Your answer clarified that you are trying to send a file. For this I would recommend sending an array of bytes[]. Using this method you can send anything that can be serialized. This includes a file. Please note that the size of the file is limited since it must be kept in memory. To write a larger file you would want to save the data in chunks as it is being streamed in.
//Please note that if the file size is large enough. It may be preferred to use a stream instead of holding the entire file in memory.
byte[] fileAsBytes = File.ReadAllBytes(fileName);
using(NetworkStream networkStream = tcpClient.GetStream())
using(BinaryReader binaryReader = new BinaryReader(networkStream))
using(BinaryWriter binaryWriter = new BinaryWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a byte array
int incomingBytesLength = BinaryReader.ReadInt32(); //The header is 4 bytes that lets us know how large the incoming byte[] is.
byte[] incomingBytes = BinaryReader.ReadBytes(incomingBytesLength);
//To send a byte array
BinaryWriter.Write(fileAsBytes.Length); //Send a header of 4 bytes that lets the listener know how large the incoming byte[] is.
BinaryWriter.Write(fileAsBytes);
}
got it working, code > 30000 chars :\
it is a little messy but hey, it's functional.
server : https://www.dropbox.com/s/2wyccxpjbja10z3/Program.cs?m
client : https://www.dropbox.com/s/yp78nx4ubacsz6f/Program.cs?m
I have a client server situation, where the client sends the data (a movie for example) to the server, the server saves that data to the HDD.
It sends the data by a fixed array of bytes. After the bytes are sent, the server asks if there is more, if yes, send more and so on. Every thing is going well, all the data gets across.
But when I try to play the movie, it cant be played and if I look to the file length of each movie (client and server) the server movie is bigger then the client movie.also when I look at the command screen at the end of the sending/receiving data there is more then a 100% of the bytes that are across.
The only thing I can think of that can be wrong is the fact that my server reads in the stream till the fixed buffer array is full and therefor has at the end more bytes then the client. However if that is the problem how can I solve this?
I've just added the 2methods of sending, because the tcp connection works, any help is welcome.
Client
public void SendData(NetworkStream nws, StreamReader sr, StreamWriter sw)
{
using (FileStream reader = new FileStream(this.path, FileMode.Open, FileAccess.Read))
{
byte[] buffer = new byte[1024];
int currentBlockSize = 0;
while ((currentBlockSize = reader.Read(buffer, 0, buffer.Length)) > 0)
{
sw.WriteLine(true.ToString());
sw.Flush();
string wait = sr.ReadLine();
nws.Write(buffer, 0, buffer.Length);
nws.Flush();
label1.Text = sr.ReadLine();
}
sw.WriteLine(false.ToString());
sw.Flush();
}
}
Server
private void GetMovieData(NetworkStream nws, StreamReader sr, StreamWriter sw, Film filmInfo)
{
Console.WriteLine("Adding Movie: {0}", filmInfo.Titel);
double persentage = 0;
string thePath = this.Path + #"\films\" + filmInfo.Titel + #"\";
Directory.CreateDirectory(thePath);
thePath += filmInfo.Titel + filmInfo.Extentie;
try
{
byte[] buffer = new byte[1024]; //1Kb buffer
long fileLength = filmInfo.TotalBytes;
long totalBytes = 0;
using (FileStream writer = new FileStream(thePath, FileMode.CreateNew, FileAccess.Write))
{
int currentBlockSize = 0;
bool more;
sw.WriteLine("DATA");
sw.Flush();
more = Convert.ToBoolean(sr.ReadLine());
while (more)
{
sw.WriteLine("SEND");
sw.Flush();
currentBlockSize = nws.Read(buffer, 0, buffer.Length);
totalBytes += currentBlockSize;
writer.Write(buffer, 0, currentBlockSize);
persentage = (double)totalBytes * 100.0 / fileLength;
Console.WriteLine(persentage.ToString());
sw.WriteLine("MORE");
sw.Flush();
string test = sr.ReadLine();
more = Convert.ToBoolean(test);
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
There is a reason why Read() returns the number of bytes read: it's possible it will return less than the size of the buffer. Because of this, you should do something like nws.Write(buffer, 0, currentBlockSize); in SendData(). But this will break your protocol, because the blocks won't have the size anymore.
But I find it hard to believe your code actually behaves the way you describe. That's because Read() in GetMovieData() also may not fill the whole buffer. Also, StreamReader is allowed to keep some data in an internal buffer, which would mean you could read some completely bogus data.
I think code like this, where you're combining Streams and StreamReaders/StreamWriters is a really bad idea. It would be hard to make it actually correct. What you should do instead is to make your protocol completely byte-based (not character-based), even if those bytes are ASCII-encoded "SEND".
Let me give it a try, but don't shoot me if it doesn't work.
I see that you have a buffer size of 1024, regardless of how many bytes there are left in the file that you send. Say you have a file of 2900 bytes, which would require to send 3 times, the last you send there will only be 852 bytes left to send. Yet, you create a buffer of 1024 and send over 1024 bytes. This means that your server receives 852 bytes of real data, and 172 zero-filled bytes. Even though, all those 172 bytes are save to the movie file on the server.
I guess there's an easy fix: When you write the data to the server, use the currentBlockSize as argument for the length. So in method SendData on the client, inside the while loop, change:
nws.Write(buffer, 0, buffer.Length);
to this:
nws.Write(buffer, 0, currentBlockSize);
Hey, I'm having an issue seperating packets using a custom binary protocol.
Currently the server side code looks like this.
public void HandleConnection(object state)
{
TcpClient client = threadListener.AcceptTcpClient();
NetworkStream stream = client.GetStream();
byte[] data = new byte[4096];
while (true)
{
int recvCount = stream.Read(data, 0, data.Length);
if (recvCount == 0) break;
LogManager.Debug(Utility.ToHexDump(data, 0, recvCount));
//processPacket(new MemoryStream(data, 0, recvCount));
}
LogManager.Debug("Client disconnected");
client.Close();
Dispose();
}
I've been watching the hex dumps of the packets, and sometimes the entire packet comes in one shot, let's say all 20 bytes. Other times it comes in fragmented, how do I need to buffer this data to be able to pass it to my processPacket() method correctly. I'm attempting to use a single byte opcode header only, should I add something like a (ushort)contentLength to the header aswell? I'm trying to make the protocol as lightweight as possible, and this system won't be sending very large packets(< 128 bytes).
The client side code I'm testing with is, as follows.
public void auth(string user, string password)
{
using (TcpClient client = new TcpClient())
{
client.Connect(IPAddress.Parse("127.0.0.1"), 9032);
NetworkStream networkStream = client.GetStream();
using (BinaryWriter writer = new BinaryWriter(networkStream))
{
writer.Write((byte)0); //opcode
writer.Write(user.ToUpper());
writer.Write(password.ToUpper());
writer.Write(SanitizationMgr.Verify()); //App hash
writer.Write(Program.Seed);
}
}
}
I'm not sure if that could be what's messing it up, and binary protocol doesn't seem to have much info on the web, especially where C# is involved. Any comment's would be helpful. =)
Solved with this, not sure if it's correct, but it seems to give my handlers just what they need.
public void HandleConnection(object state)
{
TcpClient client = threadListener.AcceptTcpClient();
NetworkStream stream = client.GetStream();
byte[] data = new byte[1024];
uint contentLength = 0;
var packet = new MemoryStream();
while (true)
{
int recvCount = stream.Read(data, 0, data.Length);
if (recvCount == 0) break;
if (contentLength == 0 && recvCount < headerSize)
{
LogManager.Error("Got incomplete header!");
Dispose();
}
if(contentLength == 0) //Get the payload length
contentLength = BitConverter.ToUInt16(data, 1);
packet.Write(data, (int) packet.Position, recvCount); //Buffer the data we got into our MemStream
if (packet.Length < contentLength + headerSize) //if it's not enough, continue trying to read
continue;
//We have a full packet, pass it on
//LogManager.Debug(Utility.ToHexDump(packet));
processPacket(packet);
//reset for next packet
contentLength = 0;
packet = new MemoryStream();
}
LogManager.Debug("Client disconnected");
client.Close();
Dispose();
}
You should just treat it as a stream. Don't rely on any particular chunking behaviour.
Is the amount of data you need always the same? If not, you should change the protocol (if you can) to prefix the logical "chunk" of data with the length in bytes.
In this case you're using BinaryWriter on one side, so attaching a BinaryReader to the NetworkStream returned by TcpClient.GetStream() would seem like the easiest approach. If you really want to capture all the data for a chunk at a time though, you should go back to my idea of prefixing the data with its length. Then just loop round until you've got all the data.
(Make sure you've got enough data to read the length though! If your length prefix is 4 bytes, you don't want to read 2 bytes and miss the next 2...)