I'm making a Remote Desktop application using TCP, I've tried/searched for many ways to capture the screen and send it but they all do the same thing.
To Send:
Capture the screen using Bitmap and the copyfrom method
Use memorystream to save the bitmap
Use TCP socket to send the bitmap serialized
To Recive:
Receive the message with readbytes method
Use memorystream to store the byte array
Use Image.FromStream(memorystream) to create a image
It works nice on LAN connection but when I connect with a remote server using VPN, the image takes 0.5 to 5 seconds to arrive
this is my code:
DeskTop Class:
internal static class Desktop
{
public static Image TakeScreenShoot()
{
Bitmap bitmap = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
try
{
using (Graphics Graphics = Graphics.FromImage(bitmap))
{
Graphics.CopyFromScreen(0, 0, 0, 0, Screen.PrimaryScreen.Bounds.Size, CopyPixelOperation.SourceCopy);
}
}
catch
{
bitmap = null;
}
return bitmap;
}
public static void SerializeScreen(Stream stream, Image Image)
{
MemoryStream memory = new MemoryStream();
Image.Save(memory, System.Drawing.Imaging.ImageFormat.Jpeg);
int numBytes = (int)memory.Length;
BinaryWriter binaryWriter = new BinaryWriter(stream);
binaryWriter.Write(numBytes);
binaryWriter.Write(memory.GetBuffer(), 0, numBytes);
stream.Flush()
}
public static Image DeserializeScreen(Stream stream)
{
BinaryReader binaryReader = new BinaryReader(stream);
int numBytes = binaryReader.ReadInt32();
byte[] buffer = binaryReader.ReadBytes(numBytes);
MemoryStream memory = new MemoryStream(buffer);
return Image.FromStream(memory);
}
}
Host class
private void SendImage()
{
while (Status == ServerStatus.Connected)
{
try
{
Image bitmap = Desktop.TakeScreenShoot();
Desktop.SerializeScreen(_NetStream,bitmap);
}
catch
{
}
}
}
Client Class
protected void ReciveMessage()
{
while(Status == ServerStatus.Connected)
{
try
{
ImageRecibed?.Invoke(Desktop.DeserializeScreen(_NetStream));
}
catch
{
}
}
}
How can I improve my code to run faster?
here a Video of the application speed
PD. I'm so new on this
I believe this is a good question. My socket programming experience is very limited. But I can think of a simple way. If you reduce the size of the byte array by compressing it before sending, the performance will probably improve slightly.
here is a good example => How to compress a Byte array without stream or system io
After the packet leaves the LAN, many external factors come into play. For example, it can make a big difference whether the VPN server you use is paid or not, or where is the location of this VPN Server? is it close to you or not?, or what is the hardware power of the vpn server you are using? There are a lot of possibilities.
Also, using Udp instead of Tcp will cause a slight increase in performance too, because the packet size of Udp is smaller and no acknowledgment. So if you use Udp and compress it before sending, maybe you can get a suitable result but the problem here is reliability. Also, even if you use Udp and compression, I'm not sure that there will be a performance increase at the level you are aiming for.
Therefore, I hope someone who is an expert on this subject will give detailed information because I am very curious about it.
One of the common limits to the performance of a TCP connection is:
Throughput <= WindowSize / RoundTripTime
If things were "fast" on your LAN but slow through the VPN, I assume the use of a VPN means greater distance, which means RoundTripTime will increase. And unless WindowSize increases accordingly, Throughput will decrease. So, what you want to do is increase WindowSize. How one does that will depend on the specifics of the OS/stack being used.
Related
I'm writing a screen mirroring app on WPF. My original code sends a bitmap over TCP from a server to a client. The original code works fine, but closes and recreates the tcp connection every time it sends a frame. This results in 30 socket open and close per second, which I assume isn't the ideal way to do it.
So I tried to rewrite it to reuse the stream each time it sends the data, but the stream starts to spit out wrong data after a while.
public void SendStream(byte[] byteArray)
{
/*
_client = IsServer ? _server.AcceptTcpClient() : new TcpClient(IP.ToString(), Port);
using (var clientStream = _client.GetStream())
{
var comp = Compress(byteArray);
clientStream.Write(comp, 0, comp.Length);
}
*/
var comp = Compress(byteArray);
_stream.Write(BitConverter.GetBytes(comp.Length), 0, 4);
_stream.Write(comp, 0, comp.Length);
}
public byte[] ReceiveStream()
{
/*
_client = IsServer ? _server.AcceptTcpClient() : new TcpClient(IP.ToString(), Port);
var stream = _client.GetStream();
return Decompress(stream);
*/
var lengthByte = new byte[4];
_stream.Read(lengthByte, 0, 4);
var length = BitConverter.ToInt32(lengthByte, 0);
var data = new byte[length];
_stream.Read(data, 0, length);
return Decompress(new MemoryStream(data));
}
Compress and Decompress function are just wrappers around the built in DeflateStream.
I have checked that the sent comp.Length and received length are the same when the error happens.
Any ideas on whats going on? Thanks. It always throws an exception after at least a few frames, never the first one (at least that I've tried so far)
(It seems to happen faster when the bitmaps are larger in size i.e. when the compression algorithm doesn't do as much cause the screen is more complicated. Not 100% sure though)
Try doing the following:
int receivedBytesCount = _stream.Read(data, 0, length);
The length variable you pass to the Read method is the maximum. The Read method may actually read less bytes than length. It will return the number of bytes it actually read. This will happen when your data is fragmented into TCP packets.
You need to keep calling Read until you receive enough bytes and combine everything to get the full frame. You will need to adjust the offset in order to avoid overwriting the buffer. In the code you posted it is hardcoded to 0.
Im currently a bit stuck with my c# project.
I have 2 applications, they both have a common class definition I call a NetMessage
a NetMessage contains a MessageType string property, as well as 2 List lists.
The idea is that I can pack this class with classes, and data to send across the network as a byte[].
Because Network Streams do not advertise the amount of data they are receiving, I modified my Send method to send the size of the NetMessage byte[] ahead of the actual byte[].
private static byte[] ReceivedBytes(NetworkStream MainStream)
{
try
{
//byte[] myReadBuffer = new byte[1024];
int receivedDataLength = 0;
byte[] data = { };
long len = 0;
int i = 0;
MainStream.ReadTimeout = 60000;
//MainStream.CanTimeout = false;
if (MainStream.CanRead)
{
//Read the length of the incoming message
byte[] byteLen = new byte[8];
MainStream.Read(byteLen, 0, 8);
len = BitConverter.ToInt64(byteLen, 0);
data = new byte[len];
//data is now set to the appropriate size for the expected message
//While we have not got the full message
//Read each individual byte and append to data.
//This method, seems to work, but is ridiculously slow,
while (receivedDataLength < data.Length)
{
receivedDataLength += MainStream.Read(data, receivedDataLength, 1);
}
//receivedDataLength += MainStream.Read(data, receivedDataLength, data.Length);
return data;
}
}
catch (Exception E)
{
//System.Windows.Forms.MessageBox.Show("Exception:" + E.ToString());
}
return null;
}
I have tried to change the size argument below to something like 1024 or to be the data.Length, but I get funky results.
receivedDataLength += MainStream.Read(data, receivedDataLength, 1);
setting it to data.Length seems to cause problems when the Class being sent is a few mb in size.
Setting the buffer size to be 1024 like I have seen in other examples, causes failures when the size of the incoming message is small, like 843 bytes, it errors out saying that I tried to read out of bounds or something.
Below is the type of method being used to send the data in the first place.
public static void SendBytesToStream(NetworkStream TheStream, byte[] TheMessage)
{
//IAsyncResult r = TheStream.BeginWrite(TheMessage, 0, TheMessage.Length, null, null);
// r.AsyncWaitHandle.WaitOne(10000);
//TheStream.EndWrite(r);
try
{
long len = TheMessage.Length;
byte[] Bytelen = BitConverter.GetBytes(len);
TheStream.Write(Bytelen, 0, Bytelen.Length);
TheStream.Flush();
// <-- I've tried putting thread sleeps in this spot to see if it helps
//I've also tried writing each byte of the message individually
//takes longer, but seems more accurate as far as network transmission goes?
TheStream.Write(TheMessage, 0, TheMessage.Length);
TheStream.Flush();
}
catch (Exception e)
{
//System.Windows.Forms.MessageBox.Show(e.ToString());
}
}
I'd like to get these two methods setup to the point where they are reliably sending and receiving data.
The application I am using this for, monitors a screenshots folder in a game directory,
when it detects a screenshot in TGA format, it converts it to PNG, then takes its byte[] and sends it to the receiver.
The receiver then posts it to Facebook (I don't want my FB tokens distributed in my client application), hence the server / proxy idea.
Its strange, but when I step through the code, the transfer is invariably successful.
But if I run it full speed, no breakpoint, it typically tells me that the connection was closed by the remote host etc.
The client typically finishes sending the data almost instantly, even though its a 4mb file.
The receiver spends about 2 minutes reading from the Network Stream, which doesnt make sense, if the client finished sending the data, does that mean the data is just floating in cyber space, and being pulled down?
Surely it should be synchronous?
I suspect I know where my code was going wrong.
It turns out that the scope I was creating my TCPClient that was doing the sending, was declared and instantiated within a method.
This being the case, the GAC was disposing of it, even though the Receiving Server had not finished downloading the stream.
I managed to resolve it by creating a method that can detect when the Client has disconnected on the server end, and until it has actually disconnected, it will keep looping/waiting until disconnected.
This way, we are waiting until the server lets go of us.
I am trying to make a small application to serve screenshot of entire screen through network. I want it to be able to serve one every 1-2 seconds through LAN. I thought it won't be a big problem, so I chose C# and nancy www self host (as the easiest option). Now, the speed is allright for my needs, but it seems to be taking way too much memory, and it grows with time. It seems to settle on taking about 3.5 GB RAM and from then no longer grows, but that's not an acceptable amount - even with big resolution (mine is 2560x1440).
Is my code bad, or is nancy not suitable for handling many big responses, or maybe C# method of capturing screen is poorly optimized and I should try pinvoke methods? Or maybe it's just a terrible way to do it and I should try something more advanced, like using VNC library?
Currently my code looks like this:
public class HomeModule : NancyModule
{
private Bitmap bitmapScreenCapture;
private Graphics graphics;
private Object lockMe = new object();
private MemoryStream memoryStream = new MemoryStream();
public HomeModule()
{
Get["image"] = parameters =>
{
lock (lockMe)
{
GC.Collect();
if (bitmapScreenCapture == null || bitmapScreenCapture.Width != Screen.PrimaryScreen.Bounds.Width || bitmapScreenCapture.Height != Screen.PrimaryScreen.Bounds.Height)
{
if (bitmapScreenCapture != null)
{
bitmapScreenCapture.Dispose();
}
bitmapScreenCapture = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
graphics = Graphics.FromImage(bitmapScreenCapture);
}
graphics.CopyFromScreen(Screen.PrimaryScreen.Bounds.X,
Screen.PrimaryScreen.Bounds.Y,
0, 0,
bitmapScreenCapture.Size,
CopyPixelOperation.SourceCopy);
bitmapScreenCapture.Save(memoryStream, ImageFormat.Png);
memoryStream.Position = 0;
return new Response()
{
Contents = stream => memoryStream.CopyTo(stream)
};
}
};
}
}
As much as possible, keep variables in the most local scope possible, and dispose of what you can.
Part of your issue may be that you're new'ing up a Graphics instance repeatedly, but never disposing of the old reference. The GC will collect it eventually, but you can place your code in a using block to let it know you're done with it.
I haven't tested this, but here I've made your instances local, and disposed of the Graphics and Bitmap instances. I didn't dispose of the MemoryStream since I'm not sure it will successfully return the value if you do, but you could play around with it.
var screen = Screen.PrimaryScreen;
using (var bitmap = new Bitmap(screen.Bounds.Width, screen.Bounds.Height))
{
using (var g = Graphics.FromImage(bitmap))
{
g.CopyFromScreen(screen.Bounds.Left, screen.Bounds.Top, 0, 0, screen.Bounds.Size);
}
var imageStream = new MemoryStream();
bitmap.Save(imageStream, ImageFormat.Png);
imageStream.Position = 0;
return new Response()
{
Contents = stream => memoryStream.CopyTo(stream)
};
}
I have the code to send and receive images via TCP that works. However, because I am sending whole images each time the bandwidth use is huge and will make my program completely unusable on slower internet connections.
To reduce the bandwidth, it is clear I only want to send the difference between the current image and the previous one. I was hoping you could provide me with some information on how to do this, or which libraries, if any, to use. I have my send and receive threads below that I am currently using to send and receive images. The use for my program is as a screen-sharing application.
Sending Image:
public void SendSS()
{
try
{
while (!mainFrm.ssStop)
{
ssTcpClient = new TcpClient();
ssTcpClient.Connect(mainFrm.contactIP, 1500);
//Set up TCP connection.
if (ssTcpClient.Connected)
{
//Connected. Capture screen image.
labelText("Connected. Now sending desktop to technician.");
Image screenShotBMP = GrabScreen();
MemoryStream ssmemStream = new MemoryStream();
screenShotBMP.Save(ssmemStream, ImageFormat.Jpeg);
NetworkStream ns = ssTcpClient.GetStream();
//Convert image to data.
byte[] bytesToSend = ssmemStream.GetBuffer();
//Store data in stream and send via port.
ns.Write(bytesToSend, 0, bytesToSend.Length);
ns.Flush();
//Dispose of image to avoid memory leakage.
screenShotBMP.Dispose();
ssTcpClient.Close();
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "SendSS()", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
Receiving Image:
public void ReceiveSS()
{
try
{
ssTcpListener = new TcpListener(IPAddress.Any, 1500);
tcpReceiver = new TcpClient();
while (!mainFrm.ssStop)
{
//Start listening for connection.
//Accept any incoming connection requests on port 1500.
ssTcpListener.Start();
tcpReceiver = ssTcpListener.AcceptTcpClient();
if (tcpReceiver.Connected)
{
//TCP connected. Receive images from contact.
labelText("Connected. Now receiving desktop from client.");
NetworkStream receivedNs = new NetworkStream(tcpReceiver.Client);
//Put image into picturebox.
Bitmap image = new Bitmap(receivedNs);
pboScrnShr.BackgroundImage = image;
receivedNs.Flush();
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "ReceiveSS()", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
Thanks for your help in advance.
After doing some more searching on the internet, I found this:
Finding difference between images - By Bryan Cook
Using that code to find the differences between the images I sent those differences across (using screenshotBMP.Save(ssMemStream, ImageFormat.Gif), rather than .JPEG), this of course save the transparency in the memory stream as black so on the receiving end you have to make sure (for the image difference that is sent) that you do receiveImage.MakeTransparency(Color.Black) otherwise if you try to overlay the difference onto the previous image you'll just get a black screen.
I managed to reduce the image received from 12Mbps down to 2Mbps, I know there is still a little more to go for slower connections though.
All credit goes to Bryan Cook for the code I just used it in a different context.
In order to develop an app remote desktop WP7, I started to with a desktop simple viewer and it works but the problem that not show all actions that I do in Server side, that's video in YouTube can show you my problem
http://www.youtube.com/watch?v=3q-FumfYsPQ&feature=youtu.be
I use socket connection and I decode and encode my data (images).
This is my code in WP7 client side
void Conncet(string IP_Address)
{
client_socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
SocketAsyncEventArgs socketEventArg = new SocketAsyncEventArgs()
{
RemoteEndPoint = new IPEndPoint(IPAddress.Parse(IP_Address), 4532)
};
socketEventArg.Completed += OnConncetCompleted;
client_socket.ConnectAsync(socketEventArg);
}
void StartReceiving()
{
byte[] response = new byte[131072];
SocketAsyncEventArgs socketEventArg = new SocketAsyncEventArgs();
socketEventArg.Completed += OnReceiveCompleted;
socketEventArg.SetBuffer(response, 0, response.Length);
client_socket.ReceiveAsync(socketEventArg);
}
private void ViewReceivedImage(byte[] buffer)
{
try
{
MemoryStream ms = new MemoryStream(buffer);
BitmapImage bi = new BitmapImage();
bi.SetSource(ms);
MyImage.Source = bi;
ms.Close();
}
catch (Exception) { }
finally
{
StartReceiving();
}
}
This is my code in Server side (PC) sending images.
void StartSending()
{
while (!stop)
try
{
Image oldimage = scr.Get_Resized_Image(wToCompare, hToCompare, scr.GetDesktopBitmapBytes());
//Thread.Sleep(1);
Image newimage = scr.Get_Resized_Image(wToCompare, hToCompare, scr.GetDesktopBitmapBytes());
byte[] buffer = scr.GetDesktop_ResizedBytes(wToSend, hToSend);
float difference = scr.difference(newimage, oldimage);
if (difference >= 1)
{
SenderSocket.Send(buffer);
}
}
catch (Exception) { }
}
My question is how can I make the send and receive fast to show the PC screen in WP7 in +/- real time.
You'll have to find what's the bottleneck and speed that up. It could be that the network is not working fast enough. Using compression might be the answer.
It could be that the WP7 machine is not fast enough to show the images. Sending partial screen images or lower resolution might be a solution.
It could be that the windows machine is simply not grabbing the images fast enough. Chancing some code around might be the solution. Using partial updates could also help.
I would go for a solution where every update I would only send one corner (upper left, upper right, lower left, lower right) so the entire stack had to send/recieve smaller blobs of data.
Be aware that the simulator might add/remove some overhead and show non realistic performance. So don't optimize the code for the simulator (too much)
I'm guessing you're running out of bandwidth.
Sending full uncompressed images is a terrible idea, if you ask me, bandwidth-wise. A single RGB32, 800x480 image is approximately 1.15 megs in size, so to sustain, let's say 15 FPS, you'd need a 138 Mbit/s connection.
I'd suggest, on the server side, to only send the rectangles that have changed, and then send them compressed. One protocol that already does this is the RFB protocol, most famously used in VNC.