I'm taking a screenshot, shrinking it down to 50% of resolution,to take less space and then I put bytes in the buffer, however unfortunately this process takes too much time.
Currently it takes from 60 to 100ms for 1920x1080 source resolution, this causes that the output video at 30fps is speeded up as I'm not producing screenshots fast enough. I need to achieve about 40ms to make video that I make out of it fluent.
Question: How can I apply a faster interpolation algorithm or limit the required operation to speed it up?
public void Screenshot(byte[] buffer)
{
using (var bmp = new Bitmap(Params.SourceWidth, Params.SourceHeight))
{
using (var g = Graphics.FromImage(bmp))
{
g.CopyFromScreen(Point.Empty, Point.Empty, new Size(Params.SourceWidth, Params.SourceHeight), CopyPixelOperation.SourceCopy);
g.Flush();
using (Bitmap resized = new Bitmap(bmp, new Size(Params.TargetWidth, Params.TargetHeight)))
{
var bits = resized.LockBits(new Rectangle(0, 0, Params.TargetWidth, Params.TargetHeight), ImageLockMode.ReadOnly, PixelFormat.Format32bppRgb);
Marshal.Copy(bits.Scan0, buffer, 0, buffer.Length);
resized.UnlockBits(bits);
}
}
}
}
This approach is slow because you're using GDI, which is no longer the main graphics API in Windows (and hasn't been since Windows Vista). One of the reasons it's slow is because it copies image data from your GPU's memory (VRAM) into your computer's main memory (RAM) for it to be manipulated with through GDI.
If you want fast operations on your computer's framebuffer, use the DWM APIs and only copy memory between buffers in your GPU. You can probably also do the image resizing and other operations really fast by doing them as shaders or CUDA programs.
Related
So I'm attempting to make a docking library very similar to the ones used in VSCode and Atom, however I've run into some issues. Basically, for the start, I need to make a box starting from width 0 to increase to the end of the screen. I'm able to achieve this:
Rectangle rectangle = new Rectangle();
myCanvas.Children.Add(rectangle);
rectangle.Width = 0;
rectangle.Height = ActualHeight;
rectangle.Fill = new SolidColorBrush(Color.FromArgb(80, 120, 120, 120));
Storyboard sb = new Storyboard();
DoubleAnimation da = new DoubleAnimation(rectangle.Width, ActualWidth, new Duration(TimeSpan.FromMilliseconds(1000)));
Storyboard.SetTargetProperty(da, new PropertyPath("(Rectangle.Width)"));
sb.Children.Add(da);
rectangle.BeginStoryboard(sb);
The animation is quite smooth on my dev-machine, but when remote-desktoping into a VM, the animation is VERY choppy. However, VSCodes and Atoms animations are still pretty decent. Is there any way to make this work better?
Thanks.
EDIT:
Is there a way to make a smooth animation for Winforms aswell?
EDIT 2:
So I have this Winforms GDI code sample here:
private void FastTimer_Tick(object sender, EventArgs e)
{
var x = 1;
if (solidBrush == IntPtr.Zero)
{
solidBrush = CreateSolidBrush(ColorTranslator.ToWin32(Color.FromArgb(120, 120, 120)));
hDC = CreateGraphics().GetHdc();
}
index += x;
int w = x;
int h = Height;
//create memory device context
var memdc = CreateCompatibleDC(hDC);
//create bitmap
var hbitmap = CreateCompatibleBitmap(hDC, index, h);
////select bitmap in to memory device context
var holdbmp = SelectObject(memdc, hbitmap);
RECT rect = new RECT(new Rectangle(0, 0, w, h));
FillRect(memdc, ref rect, solidBrush);
AlphaBlend(hDC, index - x, 0, w, h, memdc, 0, 0, w, h, new BLENDFUNCTION(0, 0, 128, 0));
SelectObject(memdc, holdbmp);
DeleteObject(hbitmap);
DeleteDC(memdc);
}
With the fast timer core being this (Interval is 1000 in testing):
double frequency = Stopwatch.Frequency;
long prevTicks = 0;
while (true)
{
if (!_enabled)
{
return;
}
double interval = frequency / Interval;
long ticks = Stopwatch.GetTimestamp();
if (ticks >= prevTicks + interval)
{
prevTicks = ticks;
Tick?.Invoke(this, EventArgs.Empty);
}
}
However, the animations are still choppy on Remote Desktop. (I know RD cannot use hardware acceleration, but I thought GDI was hardware-accelerated on compatible machines, but I do not see any GPU usage on my dev-machine)
With regards to thatguys answer, how do I make this faster?
These issues are common with WPF and are caused by bitmap remoting, see this Microsoft blog.
Starting with the release of NET 3.5 SP1 (including NET 4), WPF renders the application content using a software rasterizer on the server and then remotes the content as bitmaps in all cases.
This is crucial for animations, due to lots of redrawing. Animations often affect a large area at once, imagine a color gradient. Therefore, large parts of the application need to be invalidated and redrawn. Furthermore, there is a the lack of support for occlusion in WPF that might aggravate the issue, too.
Bitmaps are highly compressed by the underlying RDC stack and only regions that changed are being updated. Also note that WPF does not currently have efficient occlusion support, so for instance animations that are completely hidden behind other opaque WPF elements will still force invalidation and RDP update.
The reason that you might not encouter this issue for eaxmple in native Win32 or MFC applications is that they use GDI or GDI+ for drawing their user interface controls.
When apps use GDI (such as many Win32 and Winforms apps do), only the GDI primitives are remoted. In many cases this can be more efficient than remoting WPF apps since WPF apps remotes bitmaps which typically result in more content being sent over the wire than a similar GDI-based app. The additional data may result in slower performance depending on network bandwidth and the size and frequency of updates.
Atom and VS Code are built on Electron which is based on chromium, not WPF, that is why there are large differences on remote desktop.
In the linked blog you can find hints on how to improve the remote performance in general. Apart from considering to disable animations in remote desktop scenarios because there are no guarantees on factors like bandwidth or latency for your clients, the essential parts for improving animations are:
Adapt the DesiredFrameRate to reduce the animation frame rate, which in turn reduces traffic
Turn animations off in occluded areas of the application, since they cause redrawing, too
Reduce traffic by removing animations from default WPF control templates for ProgressBar, Button, ComboBox and others, as well as all of your own non-essential animations
Improve RDP compression efficiency by simplifying other areas of the application, like replacing color gradients with solid colors
Avoid anything that is slow to render in software like bitmap effects or 3D views
Use a WPF performance profiling tool like Perforator to identify the critical areas that are invalidated frequently to optimize them
I have made a tool which captures a portion of my screen.
All the pixels within that red square are loaded into a Bitmap:
private Bitmap Capture()
{
var bitmap = new Bitmap(measurementToolBounds.Width, measurementToolBounds.Height);
using (var g = Graphics.FromImage(bitmap))
{
g.CopyFromScreen(measurementToolBounds.Location, Point.Empty, measurementToolBounds.Size, CopyPixelOperation.SourceCopy);
g.Flush();
}
return bitmap;
}
Unfortunately, CopyFromScreen takes roughly 30ms to execute.
I've tried using gdi32.dll and BitBlt, too, but as far as I know, CopyFromScreen calls BitBlt behind the scenes.
Is there a faster way to do this using C# or perhaps even a 3rd party utility tool that you know of that I can use to retrieve a portion of the screen as a byte array?
I'm using a RenderTargetBitmap to render a set of controls in order to generate a PDF. The following code segment is the relevant section:
public Drawing.Image RenderPageBitmap()
{
RenderTargetBitmap bit = null;
Drawing.Bitmap bmp = null;
try
{
bit = new RenderTargetBitmap(ImageSource.PixelWidth, ImageSource.PixelHeight, 96, 96, PixelFormats.Pbgra32);
var viewBox = GetPageXaml(); //This method loads some prebuilt XAML from an embedded resource, setting the DataContext as needed.
var siz = new Size(bit.PixelWidth, bit.PixelHeight);
viewBox.Measure(siz);
viewBox.Arrange(new Rect(siz));
viewBox.UpdateLayout();
var draw = new DrawingVisual();
using (var graph = draw.RenderOpen())
graph.DrawRectangle(new BitmapCacheBrush(viewBox), null, new Rect(siz));
bit.Render(draw);
bit.Freeze();
bmp = new Drawing.Bitmap(bit.PixelWidth, bit.PixelHeight, Imaging.PixelFormat.Format32bppPArgb);
var data = bmp.LockBits(new Drawing.Rectangle(Drawing.Point.Empty, bmp.Size), ImageLockMode.WriteOnly, Imaging.PixelFormat.Format32bppPArgb);
{
bit.CopyPixels(Int32Rect.Empty, data.Scan0, data.Height * data.Stride, data.Stride);
}
bmp.UnlockBits(data);
return bmp;
}
catch (Exception)
{
bmp?.Dispose();
throw;
}
finally
{
bit?.Clear();
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
}
}
Even following other answers on StackOverflow and other forums (like clearing the bitmap and performing a garbage collection) doesn't seem to solve the issue. Each loop of this code can leak ~100MB of memory, which means I quickly hit the ~2GB limit of 32-bit processes.
The leak seems to occur on the RenderTargetBitmap.Render method exclusively, even the DrawingContext.DrawRectangle call doesn't noticeably increase memory usage.
Is there anything I can do to solve this problem?
Here's a snapshot of the memory usage as viewed through JetBrains' dotMemory. Clearly, the .Net heap is correctly cleared, but the unmanaged memory continues to grow.
You return the Bitmap to somewhere. Make sure you Dispose the Bitmap instance once you are done with it. What you are doing in the finally is useless when there is memory leak. If there are references GC wouldn't collect it.
Each loop of this code can leak ~100MB of memory, which means I
quickly hit the ~2GB limit of 32-bit processes.
Are you assuming there is a memory leak? May be there is no memory leak. I would get a good memory profiling tool and test this.
I have used ANTS Memory profiler and I find it good (it comes with 14 days trial). Just execute your logic a few times and see the Instance List if anything is growing. If so, look at the Retention graph to see what holds onto it. That will tell you what exactly happening. Root causes for memory leaks are quite difficult to guess sometimes, fortunately there are good tools for that.
Remove bit.Freeze();. Garbage collection does not collect frozen objects.
I have been checking around to convert live frames into video. And I found (NReco.VideoConverter) ffmpeg lib to convert live frames to Video, but the problem is it is taking time to write each frame to ConvertLiveMediaTask (async live media task conversion).
I have an event that provides (raw) frames (1920x1080) (25fps) from IpCamera. Whenever I get frame I am doing the following
//Image availbale event fired
//...
//...
// Record video is true
if(record)
{
//////////////############# Time taking part ##############//////////////////////
var bd = frameBmp.LockBits(new Rectangle(0, 0, frameBmp.Width, frameBmp.Height), ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
var buf = new byte[bd.Stride * frameBmp.Height];
Marshal.Copy(bd.Scan0, buf, 0, buf.Length);
// write to ConvertLiveMediaTask
convertLiveMediaTask.Write(buf, 0, buf.Length); // ffMpegTask
frameBmp.UnlockBits(bd);
//////////////////////////////////////////////////////////////////////////////////
}
As the above part is taking much time, I am loosing the frames.
//Stop recording
convertLiveMediaTask.Stop(); //ffMpegTask
Stop recording, for this part I have used BackgroundWorker, because this takes too smuch time to save the media to file.
My question is how can I write the frame to ConvertLiveMediaTask in faster way? are there any possibilites to write it in background?
Please give me suggestions.
I'm sure that most time takes encoding and compressing raw bitmaps (if you encode them with h264 or something like that) by FFMpeg because of FullHD resolution (NReco.VideoConverter is a wrapper to FFMpeg). You must know that real-time encoding of FullHD is VERY CPU consuming task; if your computer is not able to do that, you may try to play with FFMPeg encoding parameters (decrease video quality / compression ratio etc) or use encoder that requires less CPU resources.
If You need to record some limited time live stream, You can split video capturing and compressing/saving into two threads.
Use for example ConcurrentQueue to buffer live frames (En-queue) on one thread without delay, and other thread could save those frames at a pace it can (De-queue). This way you will not loose frames.
Obviously You will have strain on RAM and also after stopping Live video You will have a delay while saving thread finishes.
I have to capture a screen shot continuously after every 250 milliseconds for my program (Similar to Netmeeting). I used the following code:
Image CaptureScreenShot()
{
bmpScreenshot = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height, PixelFormat.Format32bppArgb);
gfxScreenshot = Graphics.FromImage(bmpScreenshot);
gfxScreenshot.CopyFromScreen(Screen.PrimaryScreen.Bounds.X, Screen.PrimaryScreen.Bounds.Y, 0, 0, Screen.PrimaryScreen.Bounds.Size, CopyPixelOperation.SourceCopy);
return bmpScreenshot;
}
To capture the screenshot but it slow the performance of the PC. Where, Netmeeting do not.
Is there any way to get screen shot without slowing the PC?
Code sample will be appreciated
You won't find a basic answer here. They use much more involved mechanisms for detecting changes on the screen and sending them.
Check out how terminal svcs work -
http://technet.microsoft.com/en-us/library/cc755399%28WS.10%29.aspx
ideally you are hooking into the GUI and monitoring events, etc. much more advanced than simply screen scraping. If you want to look at less advanced code check out http://www.tightvnc.com