I am trying to make a small application to serve screenshot of entire screen through network. I want it to be able to serve one every 1-2 seconds through LAN. I thought it won't be a big problem, so I chose C# and nancy www self host (as the easiest option). Now, the speed is allright for my needs, but it seems to be taking way too much memory, and it grows with time. It seems to settle on taking about 3.5 GB RAM and from then no longer grows, but that's not an acceptable amount - even with big resolution (mine is 2560x1440).
Is my code bad, or is nancy not suitable for handling many big responses, or maybe C# method of capturing screen is poorly optimized and I should try pinvoke methods? Or maybe it's just a terrible way to do it and I should try something more advanced, like using VNC library?
Currently my code looks like this:
public class HomeModule : NancyModule
{
private Bitmap bitmapScreenCapture;
private Graphics graphics;
private Object lockMe = new object();
private MemoryStream memoryStream = new MemoryStream();
public HomeModule()
{
Get["image"] = parameters =>
{
lock (lockMe)
{
GC.Collect();
if (bitmapScreenCapture == null || bitmapScreenCapture.Width != Screen.PrimaryScreen.Bounds.Width || bitmapScreenCapture.Height != Screen.PrimaryScreen.Bounds.Height)
{
if (bitmapScreenCapture != null)
{
bitmapScreenCapture.Dispose();
}
bitmapScreenCapture = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
graphics = Graphics.FromImage(bitmapScreenCapture);
}
graphics.CopyFromScreen(Screen.PrimaryScreen.Bounds.X,
Screen.PrimaryScreen.Bounds.Y,
0, 0,
bitmapScreenCapture.Size,
CopyPixelOperation.SourceCopy);
bitmapScreenCapture.Save(memoryStream, ImageFormat.Png);
memoryStream.Position = 0;
return new Response()
{
Contents = stream => memoryStream.CopyTo(stream)
};
}
};
}
}
As much as possible, keep variables in the most local scope possible, and dispose of what you can.
Part of your issue may be that you're new'ing up a Graphics instance repeatedly, but never disposing of the old reference. The GC will collect it eventually, but you can place your code in a using block to let it know you're done with it.
I haven't tested this, but here I've made your instances local, and disposed of the Graphics and Bitmap instances. I didn't dispose of the MemoryStream since I'm not sure it will successfully return the value if you do, but you could play around with it.
var screen = Screen.PrimaryScreen;
using (var bitmap = new Bitmap(screen.Bounds.Width, screen.Bounds.Height))
{
using (var g = Graphics.FromImage(bitmap))
{
g.CopyFromScreen(screen.Bounds.Left, screen.Bounds.Top, 0, 0, screen.Bounds.Size);
}
var imageStream = new MemoryStream();
bitmap.Save(imageStream, ImageFormat.Png);
imageStream.Position = 0;
return new Response()
{
Contents = stream => memoryStream.CopyTo(stream)
};
}
Related
I'm making a Remote Desktop application using TCP, I've tried/searched for many ways to capture the screen and send it but they all do the same thing.
To Send:
Capture the screen using Bitmap and the copyfrom method
Use memorystream to save the bitmap
Use TCP socket to send the bitmap serialized
To Recive:
Receive the message with readbytes method
Use memorystream to store the byte array
Use Image.FromStream(memorystream) to create a image
It works nice on LAN connection but when I connect with a remote server using VPN, the image takes 0.5 to 5 seconds to arrive
this is my code:
DeskTop Class:
internal static class Desktop
{
public static Image TakeScreenShoot()
{
Bitmap bitmap = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
try
{
using (Graphics Graphics = Graphics.FromImage(bitmap))
{
Graphics.CopyFromScreen(0, 0, 0, 0, Screen.PrimaryScreen.Bounds.Size, CopyPixelOperation.SourceCopy);
}
}
catch
{
bitmap = null;
}
return bitmap;
}
public static void SerializeScreen(Stream stream, Image Image)
{
MemoryStream memory = new MemoryStream();
Image.Save(memory, System.Drawing.Imaging.ImageFormat.Jpeg);
int numBytes = (int)memory.Length;
BinaryWriter binaryWriter = new BinaryWriter(stream);
binaryWriter.Write(numBytes);
binaryWriter.Write(memory.GetBuffer(), 0, numBytes);
stream.Flush()
}
public static Image DeserializeScreen(Stream stream)
{
BinaryReader binaryReader = new BinaryReader(stream);
int numBytes = binaryReader.ReadInt32();
byte[] buffer = binaryReader.ReadBytes(numBytes);
MemoryStream memory = new MemoryStream(buffer);
return Image.FromStream(memory);
}
}
Host class
private void SendImage()
{
while (Status == ServerStatus.Connected)
{
try
{
Image bitmap = Desktop.TakeScreenShoot();
Desktop.SerializeScreen(_NetStream,bitmap);
}
catch
{
}
}
}
Client Class
protected void ReciveMessage()
{
while(Status == ServerStatus.Connected)
{
try
{
ImageRecibed?.Invoke(Desktop.DeserializeScreen(_NetStream));
}
catch
{
}
}
}
How can I improve my code to run faster?
here a Video of the application speed
PD. I'm so new on this
I believe this is a good question. My socket programming experience is very limited. But I can think of a simple way. If you reduce the size of the byte array by compressing it before sending, the performance will probably improve slightly.
here is a good example => How to compress a Byte array without stream or system io
After the packet leaves the LAN, many external factors come into play. For example, it can make a big difference whether the VPN server you use is paid or not, or where is the location of this VPN Server? is it close to you or not?, or what is the hardware power of the vpn server you are using? There are a lot of possibilities.
Also, using Udp instead of Tcp will cause a slight increase in performance too, because the packet size of Udp is smaller and no acknowledgment. So if you use Udp and compress it before sending, maybe you can get a suitable result but the problem here is reliability. Also, even if you use Udp and compression, I'm not sure that there will be a performance increase at the level you are aiming for.
Therefore, I hope someone who is an expert on this subject will give detailed information because I am very curious about it.
One of the common limits to the performance of a TCP connection is:
Throughput <= WindowSize / RoundTripTime
If things were "fast" on your LAN but slow through the VPN, I assume the use of a VPN means greater distance, which means RoundTripTime will increase. And unless WindowSize increases accordingly, Throughput will decrease. So, what you want to do is increase WindowSize. How one does that will depend on the specifics of the OS/stack being used.
I am trying to create large number of 1 bit per pixel bmp image from base 64 string and saving. As per the requirement a very huge number of images being created in a short period of time( an average of 50,000 to 1,00,000 in a short duration). I am using the below code.
public void CreateoneBppImageAndSave(String base64ImageString,String ImagePathToSave)
{
byte[] byteArray = Convert.FromBase64String(base64ImageString);
System.Drawing.Image img = byteArrayToImage(byteArray);
Bitmap objBitmap = new Bitmap(img);
BitmapData bmpData = objBitmap.LockBits(new Rectangle(0, 0, objBitmap.Width, objBitmap.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format1bppIndexed);
Bitmap oneBppBitmap = new Bitmap(objBitmap.Width, objBitmap.Height, bmpData.Stride, System.Drawing.Imaging.PixelFormat.Format1bppIndexed, bmpData.Scan0);
oneBppBitmap.Save(ImagePathToSave, ImageFormat.Bmp);
img.Dispose();
objBitmap.Dispose();
objBitmap.UnlockBits(bmpData);
oneBppBitmap.Dispose();
}
private Image byteArrayToImage(byte[] byteArrayIn)
{
using (MemoryStream ms = new MemoryStream(byteArrayIn))
{
return Image.FromStream(ms);
}
}
Here the physical memory usage going very high. Usually the images are generated with size of 200x200 to 754x1024 . After certain duration physical memory usage reaching to the extreme and out of memory exception is being thrown.The physical memory is getting increased by 0.01 GB by every 5-10 seconds. Please help me to optimize the code in terms of memory usage.
You call LockBits on objBitmap however you call UnlockBits on oneBppBitmap. You should be calling unlock on the same object you called lock on.
As for using statements like I mentioned in the comments, a using statement turns this
using(SomeType obj = new SomeType())
{
// Some code
}
in to the equivalent of this
SomeType obj = new SomeType())
try
{
// Some code
}
finally
{
obj.Dispose();
}
That guarantees that even if a exception is thrown in // Some Code the dispose action will still happen. Your code, as it is right now, will not dispose any of its objects if any of your functions between the creation and dispose throws an exception.
Here is a re-written version with all the corrections I mentioned plus a few others..
public void CreateoneBppImageAndSave(String base64ImageString,String ImagePathToSave)
{
byte[] byteArray = Convert.FromBase64String(base64ImageString);
using(Image img = byteArrayToImage(byteArray))
using(Bitmap objBitmap = new Bitmap(img))
{
BitmapData bmpData = objBitmap.LockBits(new Rectangle(0, 0, objBitmap.Width, objBitmap.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format1bppIndexed);
try
{
using(Bitmap oneBppBitmap = new Bitmap(objBitmap.Width, objBitmap.Height, bmpData.Stride, System.Drawing.Imaging.PixelFormat.Format1bppIndexed, bmpData.Scan0))
{
oneBppBitmap.Save(ImagePathToSave, ImageFormat.Bmp);
}
}
finally
{
//put the unlock in a finally to make sure it happens.
objBitmap.UnlockBits(bmpData);
}
}
}
EDIT: If this really is in your code
objBitmap.Dispose();
objBitmap.UnlockBits(bmpData);
That is the source of your problem, you should not call any methods on a class after you dispose. That is another benefit of using, you can't call methods late because the variable goes out of scope when you leave the using block.
This code runs every 100 ms. The memory usage just keeps increasing until it hits 1.5 GB and then it crashes.
void takeScreenShot()
{
Surface s;
s = CaptureScreen();
pictureBox1.Image = new Bitmap(Surface.ToStream(s, ImageFileFormat.Bmp));
s.Dispose();
}
public Surface CaptureScreen()
{
int width = Screen.PrimaryScreen.Bounds.Width;
int height = Screen.PrimaryScreen.Bounds.Height;
Device device = new Device(new Direct3D(), 0, DeviceType.Hardware, IntPtr.Zero, CreateFlags.HardwareVertexProcessing, new PresentParameters(width, height));
DisplayMode disp = device.GetDisplayMode(0);
Surface s = Surface.CreateOffscreenPlain(device, disp.Width, disp.Height, Format.A8R8G8B8, Pool.Scratch);
device.GetFrontBufferData(0, s);
return s;
}
You are creating a new device every time.
You should create the device only once, create it in your startup code once and then keep using it.
Furthermore I suspect a memory leak in Surface.ToStream() the returned stream probably needs disposing too.
var stream = Surface.ToStream(s, ImageFileFormat.Bmp);
pictureBox1.Image = new Bitmap(stream);
stream.Dispose();
As Hans Passant mentioned, Bitmap needs disposing as well.
You can very easily debug memory leaks in SharpDX by a helper to make diagnostic about unreleased COM resources. Setup at the beginning of your application this variable:
SharpDX.Configuration.EnableObjectTracking = true;
When your application exit, It will a print a report of COM objects that were not properly released with the stacktrace. The class behind this is ObjectTracker.
ObjectTracker.ReportActiveObjects() can be called to print the currently used resources at runtime (even with stack trace).
I'm working on a project using WPF to display the Kinect ColorImageFrame and a skeleton representation. I also have to record those two videos.
I'm able to display and record (using EmguCV) those two images, but I have some performance issues. It seems that this part of my code is the reason of my loss of performance.
private void DrawSkeleton(Skeleton[] skeletons)
{
using (System.Drawing.Bitmap skelBitmap = new System.Drawing.Bitmap(640, 480))
{
foreach (Skeleton S in skeletons)
{
if (S.TrackingState == SkeletonTrackingState.Tracked)
{
DrawBonesAndJoints(S,skelBitmap);
}
else if (S.TrackingState == SkeletonTrackingState.PositionOnly)
{
}
}
_videoArraySkel.Add(ToOpenCVImage<Bgr, Byte>(skelBitmap));
BitmapSource source = ToWpfBitmap(skelBitmap);
this.skeletonStream.Source = source;
}
}
and more precisely from the ToWpfBitmap which allows me to display it in my Window:
public static BitmapSource ToWpfBitmap(System.Drawing.Bitmap bitmap)
{
using (MemoryStream stream = new MemoryStream())
{
bitmap.Save(stream, System.Drawing.Imaging.ImageFormat.Bmp);
stream.Position = 0;
BitmapImage result = new BitmapImage();
result.BeginInit();
// According to MSDN, "The default OnDemand cache option retains access to the stream until the image is needed."
// Force the bitmap to load right now so we can dispose the stream.
result.CacheOption = BitmapCacheOption.OnLoad;
result.StreamSource = stream;
result.EndInit();
result.Freeze();
return result;
}
}
The loss of performance is characterized by:
- The videos displayed on the Window are not fluent anymore
- The video recording seems to miss some frames which leads to a video going faster/lower than the normal.
Can you help me by telling me where this problem may come from?
Try to use RecyclableMemoryStream instead of MemoryStream. It was designed for solving some issue with memory.
Check out this article for details - Announcing Microsoft.IO.RecycableMemoryStream
Have you tried doing the memory write i/o in a separate thread, while maintaining the data in a buffer like a queue?
I'm developing an application that uses a mobile device to take a photo and send it using a webservice. But after I've taken 4 photos I am getting an OutOfMemoryException in the code below. I tried calling GC.Collect() but it didn't help either. Maybe someone here could be give me an advice how to handle this problem.
public static Bitmap TakePicture()
{
var dialog = new CameraCaptureDialog
{
Resolution = new Size(1600, 1200),
StillQuality = CameraCaptureStillQuality.Default
};
dialog.ShowDialog();
// If the filename is empty the user took no picture
if (string.IsNullOrEmpty(dialog.FileName))
return null;
// (!) The OutOfMemoryException is thrown here (!)
var bitmap = new Bitmap(dialog.FileName);
File.Delete(dialog.FileName);
return bitmap;
}
The function is called by an event handler:
private void _pictureBox_Click(object sender, EventArgs e)
{
_takePictureLinkLabel.Visible = false;
var image = Camera.TakePicture();
if (image == null)
return;
image = Camera.CutBitmap(image, 2.5);
_pictureBox.Image = image;
_image = Camera.ImageToByteArray(image);
}
I suspect you are holding onto references. As a minor cause, note that dialogs don't dispose themselves when using ShowDialog, so you should be using the dialog (although I would expect GC to still collect an undisposed but non-referenced dialog).
Likewise, you should probably be using the image, but again: not sure I'd expect this to make-or-break; worth a try, though...
public static Bitmap TakePicture()
{
string filename;
using(var dialog = new CameraCaptureDialog
{
Resolution = new Size(1600, 1200),
StillQuality = CameraCaptureStillQuality.Default
}) {
dialog.ShowDialog();
filename = dialog.FileName;
}
// If the filename is empty the user took no picture
if (string.IsNullOrEmpty(filename))
return null;
// (!) The OutOfMemoryException is thrown here (!)
var bitmap = new Bitmap(filename);
File.Delete(filename);
return bitmap;
}
private void _pictureBox_Click(object sender, EventArgs e)
{
_takePictureLinkLabel.Visible = false;
using(var image = Camera.TakePicture()) {
if (image == null)
return;
image = Camera.CutBitmap(image, 2.5);
_pictureBox.Image = image;
_image = Camera.ImageToByteArray(image);
}
}
I'd also be a little cautious of the CutBitmap etc, to ensure that things are released ASAP.
Your mobile device usually does not have any memory swapping to disk option, so since you choose to store your images as bitmaps in memory rather than files on disk, you quickly consume your phone's memory. Your "new Bitmap()" line allocates a large chunk of memory, so it is very likely to throw the exception there. Another contender is your Camera.ImageToByteArray that will allocate a large amount of memory. This probably isn't large to what you're used to with your computer, but for your mobile this is gigantic
Try keeping the pictures on disk until you use them, i.e. until sending them to the webservice. For displaying them, use your built-in controls, they are probably the most memory efficient and you can usually point them to the image files.
Cheers
Nik