vtkWindowToImageFilter causing memory leak? - c#

I have written a small tool that creates screenshots of a number of STLs from different camera angles. vtkWindowToImageFilterconverts the contents of the vtkWin32OpenGLRenderWindowto an image, after which vtkPngWriteris used to write the image to a file.
I have noticed that the memory usage of the tool keeps rising with every screenshot it makes. After some debugging, I found out that the line of code that causes a rise in memory that is not freed at some point is
windowToImageFilter.Update();
Does anyone know if I should free some memory used by vtkWindowToImageFilter?
This is the complete function that creates the screenshots from different camera angles, for one set of STLs.
public void makeImage(vtkRenderWindow window, vtkRenderer renderer, string filePrefix="")
{
if (_parts.Count == 0)
return;
// Make cameras
Cameras cameras = new Cameras(_axial, _coronal, _sagittal, _parts[0].center, _parts[0].zoomfactor);
// VTK
// Setup vtkPNGwriter
vtkPNGWriter writer = vtkPNGWriter.New();
// Setup window to image filter
vtkWindowToImageFilter windowToImageFilter = vtkWindowToImageFilter.New();
windowToImageFilter.SetInput(window);
windowToImageFilter.SetMagnification(2); // set the resolution of the output image (3 times the current resolution of vtk render window)
foreach (string camName in cameraNames)
{
vtkCamera cam = cameras.getCameraByName(camName);
if (cam == null)
continue;
// Skip already created screenshots
if (File.Exists(getFilename(camName,filePrefix)))
continue;
// Clear renderer
renderer.RemoveAllViewProps();
// Show parts
this.show(renderer);
// Set camera
renderer.SetActiveCamera(cam);
renderer.GetActiveCamera().SetClippingRange(1.0, 1000);
window.Render();
// Update window to image filter
windowToImageFilter.Update();
// Write to png
writer.SetFileName(tempFile);
writer.SetInput(windowToImageFilter.GetOutput());
writer.Write();
// Crop png image
Bitmap img = new Bitmap(tempFile);
Bitmap imgCrop = cropImage(img);
imgCrop.MakeTransparent(Color.White);
imgCrop.Save(getFilename(camName,filePrefix));
// Dispose images (necessary to avoid memory filling)
img.Dispose();
imgCrop.Dispose();
// Delete temporary file
File.Delete(tempFile);
}
}

Because I noticed that the tool created several screenshots from the same camera viewpoint (caused by the SetInput method of vtkWindowToImageFilter being used outside the loop), I rewrote the code to create a vtkWindowToImageFilter and a vtkPngWriter inside the foreach loop. This allowed me to dispose them after they have been used, freeing up the memory.
Note that in the previous version of the code, I also tried to dispose the vtkWindowToImageFilter and vtkPngWriter right after the end of the loop, but this did not seem to free up any memory. Unfortunately, it is not clear to me why this would not work.
The current version of the code looks like this and does not cause a memory leak, as stated before.
public void makeImage(vtkRenderWindow window, vtkRenderer renderer,string filePrefix="")
{
if (_parts.Count == 0)
return;
// Make cameras
Cameras cameras = new Cameras(_axial, _coronal, _sagittal, _parts[0].center, _parts[0].zoomfactor);
foreach (string camName in cameraNames)
{
vtkCamera cam = cameras.getCameraByName(camName);
if (cam == null)
continue;
// Skip already created screenshots
if (File.Exists(getFilename(camName, filePrefix)))
continue;
// Clear renderer
renderer.RemoveAllViewProps();
// Show parts
this.show(renderer);
// Set camera
renderer.SetActiveCamera(cam);
renderer.GetActiveCamera().SetClippingRange(1.0, 1000);
window.Render();
// Update window to image filter
vtkWindowToImageFilter windowToImageFilter = vtkWindowToImageFilter.New();
windowToImageFilter.SetMagnification(4); // set the resolution of the output image (3 times the current resolution of vtk render window)
windowToImageFilter.SetInput(window);
// Write to png
vtkPNGWriter writer = vtkPNGWriter.New();
writer.SetInputConnection(windowToImageFilter.GetOutputPort());
writer.SetFileName(tempFile);
writer.Write();
// Dispose pipeline parts
windowToImageFilter.Dispose();
writer.Dispose();
// Crop png image
Bitmap img = new Bitmap(tempFile);
Bitmap imgCrop = cropImage(img, squareImages);
imgCrop.MakeTransparent(Color.White);
imgCrop.Save(getFilename(camName,filePrefix));
// Dispose images (necessary to avoid memory filling)
img.Dispose();
imgCrop.Dispose();
// Delete temporary file
File.Delete(tempFile);
}
}

Related

imagesharp trying to make an animated gif that count from 0 to 9

anyone could tell me why this generate a black only animated gif?
the code also output each in memory generated gif to show that they are different
public static void Test()
{
Image<Rgba32> img = null;
Image<Rgba32> gif = null;
TextGraphicsOptions textGraphicsOptions = new TextGraphicsOptions(true);
SolidBrush<Rgba32> brushYellow = new SolidBrush<Rgba32>(Rgba32.Yellow);
FontCollection fonts = new FontCollection();
fonts.Install(fontLocation);
Font font = fonts.CreateFont("Liberation Mono", PngFontHeight, FontStyle.Regular);
gif = new Image<Rgba32>(400, 400);
for (int i = 0; i < 10;++i)
{
img = new Image<Rgba32>(400, 400);
img.Mutate(x => x.Fill(Rgba32.Black));
img.Mutate(x => x.DrawText(textGraphicsOptions, i.ToString(), font, brushYellow, new PointF(1,1)));
gif.Frames.AddFrame(img.Frames[0]);
using (FileStream fs = File.Create(Path.Join(Program.workingDirectory, string.Format("Test-{0}.gif", i))))
{
img.SaveAsGif(fs);
}
img.Dispose();
}
using (FileStream fs = File.Create(Path.Join(Program.workingDirectory, "Test.gif")))
{
gif.SaveAsGif(fs);
}
}
if I code it to load each individual physical file using this code it make the animated gif as expected.
I want to create the animated gif in memory only.
When you create an Image<>...
gif = new Image<Rgba32>(400, 400);
...gif.Frames[0] is a "transparent black" frame (each pixel's RGBA value is #00000000). The additional frames you create in your for loop and add with...
gif.Frames.AddFrame(img.Frames[0]);
...become gif.Frames[1] through gif.Frames[10], for a total of 11 frames.
The GIF encoder uses GifColorTableMode to decide if a color table is generated for each frame or the color table for the first frame is used for all frames. The combination of the default value, GifColorTableMode.Global, plus that first transparent frame results in an 11-frame .gif file with only one color, that same "transparent black". This is why your yellow text doesn't appear and every frame looks the same.
To solve this, at some point before you save the file you need to remove that initial transparent frame so it doesn't influence color table calculations and because it's not part of your animation, anyways...
gif.Frames.RemoveFrame(0);
You may also want to change to GifColorTableMode.Local so your .gif file contains color tables reflecting all of the colors rendered...
gif.MetaData.GetFormatMetaData(GifFormat.Instance).ColorTableMode = GifColorTableMode.Local;
...although your 10 frames each use almost the same set of colors so if file size is a greater concern than color representation you might leave that property alone. Generating your 400 × 400 animation with GifColorTableMode.Global produces a 9,835-byte file whereas GifColorTableMode.Local produces a 16,703-byte file; 70% bigger but I can't tell the difference between them.
Related issue on GitHub
By the way, since I found this along the way, if you wanted to change the duration of the animated frames you would do so using another GetFormatMetaData() method similar to the one shown above...
GifFrameMetaData frameMetaData = img.MetaData.GetFormatMetaData(GifFormat.Instance);
frameMetaData.FrameDelay = 100;// 1 second

How can I make an offline png sequence recorder at 60fps?

I want to capture a sequence of images, 60fps, 4k, of the Unity Camera in non-realtime (offline).
So far I am able to capture each frame successfully, but I can't figure out how to make it offline, and conform to 60fps.
I am capturing the frames like this:
// get main camera and manually render scene into rt
Camera camera = this.GetComponent<Camera>(); // NOTE: added because there was no reference to camera in original script; must add this script to Camera
camera.targetTexture = renderTexture;
camera.Render();
this.lastRenderTime = Time.realtimeSinceStartup;
// read pixels will read from the currently active render texture so make our offscreen
// render texture active and then read the pixels
RenderTexture.active = renderTexture;
screenShot.ReadPixels(rect, 0, 0);
// reset active camera texture and render texture
camera.targetTexture = null;
RenderTexture.active = null;
// get our unique filename
string filename = uniqueFilename((int)rect.width, (int)rect.height);
// pull in our file header/data bytes for the specified image format (has to be done from main thread)
byte[] fileHeader = null;
byte[] fileData = null;
// create new thread to save the image to file (only operation that can be done in background)
new System.Threading.Thread(() =>
{
// create file and write optional header with image bytes
var f = System.IO.File.Create(filename);
if (fileHeader != null) f.Write(fileHeader, 0, fileHeader.Length);
f.Write(fileData, 0, fileData.Length);
f.Close();
Debug.Log(string.Format("Wrote screenshot {0} of size {1}", filename, fileData.Length));
}).Start();
Can someone point me in the right direction?
In time settings, set all timestep to 1/60 (approx 0.01666667)
Then caputre screen in OnPostRender event.

Reversed bitmap when creating AVI container

I have a list of images in my program, and I am generating an AVI video from them. For that purpose I use avifilewrapper_src library that handles the creation of video.
The process of creating is:
Bitmap bitmap;
//load the first image
bitmap = (Bitmap)imageSequence[0];
//create a new AVI file
AviManager aviManager = new AviManager(paths.outputVideo, false);
//add a new video stream and one frame to the new file
VideoStream aviStream =
aviManager.AddVideoStream(true, (double)nud_picturePerSec.Value, bitmap);
if(chb_audio.Checked)
aviManager.AddAudioStream(paths.sampleAudio, 0);
int count = 0;
for (int n = 0; n < imageSequence.Count; n++) {
bitmap = (Bitmap)imageSequence[n];
aviStream.AddFrame(bitmap);
bitmap.Dispose();
count++;
}
aviManager.Close();
If I keep giving different images, it works fine. If I however, put two similar images, than the video shows second image upside down (left/right side is correct). By two similar images I mean creating second image and copying it from the first one.
I have a feeling that this is somehow related to streams, but I can't find why the images are inverted.
Well I didn't managed to find the cause of that behavior. But fliping it between each use does the correction well.
bitmap.RotateFlip(RotateFlipType.RotateNoneFlipY);

What's wrong with my bitmap? How to get pixel data of what I drew?

Here's my buffer for an animation:
Bitmap PixBuffer;
Here's how I create it:
PixBuffer = new Bitmap(ClientRectangle.Width, ClientRectangle.Height, PixelFormat.Format32bppArgb);
Here's how I draw on it:
Graphics Renderer { get { return Graphics.FromImage(PixBuffer); } }
To make long story short. It works. I draw. I see changes. I use the bitmap as BackgroundImage for a window. Since the window has DoubleBuffered = true, it's silky smooth and fast.
OK, and the WTF part. I try to clone a slice of my bitmap, or even whole thing:
PixBuffer = (Bitmap)PixBuffer.Clone();
It doesn't make much sense, it should do nothing with what's displayed. But guess what - the clone is EMPTY! Exactly the same result if I try to draw PixBuffer on a new bitmap. The contents of PixBuffer is displayed. It can be even stretched as windows background. But I see no way to copy it. RotateFlip has no effect too.
What am I doing wrong? How to get pixel data of what I drew?
Freeze = (Bitmap)PixBuffer.Clone();
using (var g = Graphics.FromImage(Freeze)) {
g.FillRectangle(BrushF, 0, 0, 100, 100);
g.CompositingMode = CompositingMode.SourceOver;
g.Dispose();
}
var test = (Bitmap)Freeze.Clone();
BackgroundImage = test;
When I set PixBuffer as BackgroundImage - I get my drawn image. When I set Freeze as BackgroundImage - I get a square.
Then, if I clone Freeze to let's say Freeze1 - I still get my square, so cloning actually works on some bitmaps. But on PixBuffer NO JOY!
PixBuffer is not drawn in one frame. It is drawn as progressing animation during ca 30s. After animation completes - I have still screen - this screen I want to have as a normal bitmap to manipulate (like scaling and such). It seems like PixBuffer is write-only. I can still draw on it, but I can't copy anything from it.
I even tried to convert it to Icon and then back to Bitmap - but it's exactly the same like I was doing operations on empty Bitmap object.
But it IS NOT EMPTY! I tested it. I removed the BackgroundImage. I set another image in its place. And then I set PixBuffer as BacgroundImage again - and it is not empty, there is all I drew.
I'm missing something.
It's one of those very nasty bugs in code.
void RenderFrame(object sender, EventArgs e) {
using (var r = Graphics.FromImage(PixBuffer)) r.Clear(Color.Transparent);
var end = false;
for (var i = 0; i < Speed; i++) if (end = !UnmaskOne()) break;
RenderText();
RenderPattern();
if (end) FreezeContent = true;
Refresh();
}
I tried to copy my bitmap in UnmaskOne() method, which is called directly after clearing the frame, since this method can detect, if there's nothing more to unmask. However I had to wait with copying the bitmap - it should be drawn first with RenderText() and RenderPattern() methods. No magic here. Just plain human error.

IP Camera stop streaming after some time

I am working on one application where I want to use IP camera for displaying video streaming and and some other major operations on image captured by the IP Camera.
Libraries used in Camera capture
For Camera Capture : Emgu.CV Library
Below is the code which I am using in C#.
Variable Declaration
private Capture capture; //takes images from camera as image frames
private Emgu.CV.UI.ImageBox img; // Dynamic Picture Controls
private int nCam; // no of cameras
Code for Processing Image
private void ProcessFrame(object sender, EventArgs arg)
{
try
{
// Live Streaming Display
Image<Bgr, Byte> ImageFrame = capture.QueryFrame();
// If Ip camera try to reinitialize the IP camera
if(ImageFrame == null)
{
capture.Dispose();
capture = new Capture(URL);
ImageFrame = capture.QueryFrame();
}
ImageFrame = ImageFrame.Resize(img.Width, img.Height, Emgu.CV.CvEnum.INTER.CV_INTER_LINEAR);
img.Image = ImageFrame;
// Here I am doing some other operations like
// 1. Save Image captured from the IP Camera
// 2. Detect faces in Image
// 3. Draw Face markers on Image
// 4. Some database based on result of Face Detection
// 4. Delete image File
// continue Looping for other Ip Cameras
}
catch (NullReferenceException e)
{
}
}
Now, The Problem is after some time the QueryFrame() provide null value and camera Stop streaming.
Can any one tell me why this is happening?
How I can resolve this problem?
If any more information is needed Please Let me know.
Thanks in Advance.
Sorry about the delay but I have provide an example that works with several public IP cameras. It will need the EMGU reference replacing with your current version and the target build directory should be set to "EMGU Version\bin" alternatively extract it to the examples folder.
http://sourceforge.net/projects/emguexample/files/Capture/CameraCapture%20Public%20IP.zip/download
Rather than using the older QueryFrame() method it uses the RetrieveBgrFrame() method. It has worked reasonably well and I have had no null exceptions. However if you do replace the ProcessFrame() method with something like this
You should not be attempting to do any operations if the frame returned is Image is a nullable field and should not have a problem if _capture.RetrieveBgrFrame(); returns null if there is a problem then there is a bigger issue.
private void ProcessFrame(object sender, EventArgs arg)
{
//If you want to access the image data the use the following method call
//Image<Bgr, Byte> frame = new Image<Bgr,byte>(_capture.RetrieveBgrFrame().ToBitmap());
if (RetrieveBgrFrame.Checked)
{
Image<Bgr, Byte> frame = _capture.RetrieveBgrFrame();
//because we are using an autosize picturebox we need to do a thread safe update
if(frame!=null)
{
DisplayImage(frame.ToBitmap());
Image<Bgr, Byte> ImageFrame = frame.Resize(img.Width, img.Height, Emgu.CV.CvEnum.INTER.CV_INTER_LINEAR);
// Here I am doing some other operations like
// 1. Save Image captured from the IP Camera
// 2. Detect faces in Image
// 3. Draw Face markers on Image
// 4. Some database based on result of Face Detection
// 4. Delete image File
// continue Looping for other Ip Cameras
}
//else do nothing as we have no image
}
else if (RetrieveGrayFrame.Checked)
{
Image<Gray, Byte> frame = _capture.RetrieveGrayFrame();
//because we are using an autosize picturebox we need to do a thread safe update
if (frame != null) DisplayImage(frame.ToBitmap());
}
}
On a separate note your comment 'continue Looping for other Ip Cameras' may cause several issues. You should have a new Capture constructor for each camera camera you are using. How many camera are you using? and what public ip camera are you using so I can attempt to replicate the issue? The reason for the separate constructor is that ip cameras take a while to negotiate connections with and constantly Disposing of the original construct and replacing it will play havoc with the garbage collector and introduce no end if timing issues.
Cheers
Chris
[EDIT]
If your camera is returning null frames after a timeout period then I would check to see if there is an issue with the setup or maybe your connection is so slow it disconnects you to reduce lag to others there are various causes but this is not a code problem. You can use c# alone to acquire the data to a bitmap and then pass this to an Image type variable. There is a great article here:
http://www.codeproject.com/Articles/15537/Camera-Vision-video-surveillance-on-C
I've adapted this so you can use a HttpWebRequest as a final check to see if the stream is alive although there are still null exceptions that will be produced here:
using System.Net;
using System.IO;
string url;
private void ProcessFrame(object sender, EventArgs arg)
{
//***If you want to access the image data the use the following method call***/
//Image<Bgr, Byte> frame = new Image<Bgr,byte>(_capture.RetrieveBgrFrame().ToBitmap());
if (RetrieveBgrFrame.Checked)
{
Image<Bgr, Byte> frame = _capture.RetrieveBgrFrame();
//because we are using an autosize picturebox we need to do a thread safe update
if (frame != null)
{
DisplayImage(frame.ToBitmap());
}
else
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
// get response
WebResponse resp = req.GetResponse();
//get stream
Stream stream = resp.GetResponseStream();
if (!stream.CanRead)
{
//try reconnecting the camera
captureButtonClick(null, null); //pause
_capture.Dispose();//get rid
captureButtonClick(null, null); //reconnect
}
}
}
else if (RetrieveGrayFrame.Checked)
{
Image<Gray, Byte> frame = _capture.RetrieveGrayFrame();
//because we are using an autosize picturebox we need to do a thread safe update
if (frame != null) DisplayImage(frame.ToBitmap());
}
}
private void captureButtonClick(object sender, EventArgs e)
{
url = Camera_Selection.SelectedItem.ToString(); //add this
... the rest of the code
}
To display multiple webcams you would create a class to handle the Capture construct and processframe event. Ideally you would raise an purpose built event call that would include a camera identifier as the frameready event call does not call this. I have to make things easier created a form with as a MDI parent and opened an object to manage the capture variables and frame ready event. The Alpha version is available here:
http://sourceforge.net/projects/emguexample/files/Capture/CameraCapture%20Public%20IP%20Multipl%20Display%20Alpha.zip/download

Categories

Resources