Develop Reference

How To Extract specific Frame using AForge Library

I'm trying to extract a specific frame from a video file. I have a frame that I want when I play a video file with the aforge library. I call a new frame event, and if the new frame matches my specific frame, then it shows me a message: "Frame Match". This specific frame randomly appears in a video file. Here is my code:
private void Form1_Load(object sender, EventArgs e)
{
IVideoSource videoSource = new FileVideoSource(#"e:\media\test\a.mkv");
playerControl.VideoSource = videoSource;
playerControl.Start( );
videoSource.NewFrame += new AForge.Video.NewFrameEventHandler(Video_NewFrame );
}
private void Video_NewFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
{
//Create Bitmap from frame
Bitmap FrameData = new Bitmap(eventArgs.Frame);
//Add to PictureBox
pictureBox1.Image = FrameData;
//compare current frame to specific fram
if (pictureBox1.Image == pictureBox2.Image)
{
MessageBox.Show("Frame Match");
}
}
pictureBox2.image is a fixed frame that I want to match. This code is working fine when I play video files and extract new frames, but I am unable to compare new frames to specific frames. Please guide me on how to achieve this.

You can take a look at:
https://github.com/dajuric/accord-net-extensions
var capture = new FileCapture(#"C:\Users\Public\Videos\Sample Videos\Wildlife.wmv");
capture.Open();
capture.Seek(<yourFrameIndex>, SeekOrigin.Begin);
var image = capture.ReadAs<Bgr, byte>();
or you can use standard IEnumerable like:
var capture = new FileCapture(#"C:\Users\Public\Videos\Sample Videos\Wildlife.wmv");
capture.Open();
var image = capture.ElementAt(<yourFrameIndex>); //will actually just cast image
Examples are included.
moved to: https://github.com/dajuric/dot-imaging

As far as I can understand your problem the issue is that you can't compare image to image this way. I think you will find that the way to do this is to build a histogram table and then compare image histograms.
Some of the related things to look into are:
how to compare two images
image comparer class form VS 2015 unit testing
The second one is from unit testing library so not sure of performance (haven't tried myself yet)

Picturebox getting big red X but I can't detect or fix it

I am currently working with AForge, and have an on new frame event that posts the frame, as a bitmap, into a picturebox. 90% of the time it works great... UNLESS I fiddle with something on the winform. Changing a combo box, moving the window, or anything like that risks causing the Picturebox to switch from the video to a big red X. Code sample below:
private void connectButton_Click(object sender, EventArgs e)
{
try
{
cam = new VideoCaptureDevice(captureDevices[CameraSelectComboBox.SelectedIndex].MonikerString);
cam.NewFrame -= Handle_New_Frame; //Just to avoid the possibility of a second event handler being put on
cam.NewFrame += new AForge.Video.NewFrameEventHandler(Handle_New_Frame);
cam.Start();
}
catch
{
MessageBox.Show("An error has occured with connecting to the specified webcam. The application will now close!");
Application.Exit();
}
}
private void Handle_New_Frame(object sender, NewFrameEventArgs eventArgs)
{
try
{
if (bitmap != null)
bitmap.Dispose(); //Without this, memory goes nuts
bitmap = new Bitmap(eventArgs.Frame);
}
catch { }
//Draw some stuff on the images
bitmap = AdjustBrightness(bitmap, brightnessMeter);
bitmap = ApplyContrast(contrastMeter, bitmap);
bitmap = Draw_Top_Line(bitmap);
bitmap = Draw_Bottom_Line(bitmap);
//Set the image into the picturebox
this.Invoke((MethodInvoker)delegate
{
videoPictureBox1.Image = bitmap;
frameRate++; //Keep track of the frame rate
});
GC.Collect(); //Without this, memory goes nuts
this.Invoke((MethodInvoker)delegate {
videoPictureBox1.Refresh(); //NOT NECESSARY. JUST TRYING TO FIX THE BIG RED X!
});
if (videoPictureBox1.Image == videoPictureBox1.ErrorImage)
{
cam.Stop(); //ALSO NOT NECESSARY> AGAIN, JUST TRYING TO FIX THE BIG RED X!
cam.Start();
}
}
I put a break on the if (videoPictureBox1.Image == videoPictureBox1.ErrorImage) and it is evaluating to false, even when the big red X is up, because the image is actually being set to the bitmap. So cam.Stop() and cam.Start() never run (not sure if that would even help, but I figured I would give it a try).
videoPictureBox1.Refresh() is running every time, but again- it's not making a difference. Still have the big red X.
As I said before: if I start the video and touch nothing, the big red X will never happen. But the moment I start changing combo boxes, or dragging the form itself around, the chance of the big red X goes up exponentially. Sometimes I can flip through the combo box 10-12 times before it happens, other times it happens the second I click the combobox. :-\
Can anyone explain what is happening here and perhaps offer a suggestion on the best method to go about fixing it? I'm still very new to threading, so I've been struggling to wrap my head around exactly what is happening here and the best way to fix the issue! Any nudges in the right direction would be a huge help!

In the end, I wrapped EVERYTHING in in the Handle_New_Frame in an invoke. It completely removed the big red X issue, permanently. >_>
private void Handle_New_Frame(object sender, NewFrameEventArgs eventArgs)
{
this.Invoke((MethodInvoker)delegate
{
try
{
if (bitmap != null)
{
bitmap.Dispose(); //Without this, memory goes nuts
}
bitmap = new Bitmap(eventArgs.Frame);
}
catch { }
//Draw some stuff on the images
bitmap = AdjustBrightness(bitmap, brightnessMeter);
bitmap = ApplyContrast(contrastMeter, bitmap);
bitmap = Draw_Top_Line(bitmap);
bitmap = Draw_Bottom_Line(bitmap);
//Set the image into the picturebox
this.Invoke((MethodInvoker)delegate
{
videoPictureBox1.Image = bitmap;
frameRate++; //Keep track of the frame rate
});
GC.Collect(); //Without this, memory goes nuts
});
}

Shawn Hargreaves has an excellent, concise writeup of the "big red X of doom". I found it very helpful in the general case of dealing with WinForm components suddenly showing the red "X".
In summary:
This is caused by a control throwing an exception out of the OnPaint event.
Once it is thrown, that control will continue to show the red X and skip firing OnPaint.
To debug, set the debugger to catch Common Language Runtime Exceptions, and then do whatever you normally do to get the red X. The debugger will stop right where it is happening, allowing you to investigate and hopefully figure out a way to prevent it.

try using the clone in places where you use the bitmap.
Ex:
videoPictureBox1.Image = (Bitmap)bitmap.Clone();

Just to sum it up, this is a tested bare minimum that shows no red cross for me, even when resized, started, stoped or resolution changed.
public partial class PictureBoxVideo : Form
{
public PictureBoxVideo()
{
InitializeComponent();
var videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
var videoSource = new VideoCaptureDevice(videoDevices[0].MonikerString);
videoSource.NewFrame += Handle_Very_New_Frame;
videoSource.Start();
}
private void Handle_Very_New_Frame(object sender, NewFrameEventArgs eventArgs)
{
this.Invoke((MethodInvoker)delegate {
pictureBox.Image = new Bitmap(eventArgs.Frame);
});
}
}
Please note, that we call videoSource.Start(); from the GUI-(creation)-thread, but the call back handler (Handle_Very_New_Frame) is called from the video (worker) thread.
I think that's why we need both, the Invoke and the new Bitmap, so the new bmp will also be generated from the gui-thread. But I'm just guessing here, as I couldn't come up with a proof.

Imitating MSpaints drawing methods

For graphics exercises and some self-improvement sorta stuff, i've decided to basically just mess around and try and recreate some of the functionality of paint within a winform. I've got a lot of the standard ones to work, for example paint can, drawing dots around the cursor, free hand draw and what not, however i'm a little puzzled as to how paint does the mid-drawing animations. For example;
To draw a simple line i can simply get mouse coordinates on MouseUp and MouseDown events, and use the graphics class to draw a line between the two.
However, on MSpaint whilst drawing a line, you get almost a "preview" of the line after you click the first point, and whilst dragging it to the second point the line follows your cursor, but i'm a little stuck as to how this would be done? Does it involve constant redrawing of the line and the graphics device? It'd be great if someone could give me some hints / inner knowledge, i've had a search around the internet but can't REALLY find anything of use..

And very modern via ControlPaint.DrawReversibleLine method :)
Point? startPoint;
Point? endPoint;
private void Form_MouseDown(object sender, MouseEventArgs e)
{
startPoint = PointToScreen(e.Location);
}
private void Form_MouseMove(object sender, MouseEventArgs e)
{
if (!startPoint.HasValue)
return;
if (endPoint.HasValue)
ControlPaint.DrawReversibleLine(startPoint.Value, endPoint.Value, Color.White);
endPoint = PointToScreen(e.Location);
ControlPaint.DrawReversibleLine(startPoint.Value, endPoint.Value, Color.White);
}
private void Form_MouseUp(object sender, MouseEventArgs e)
{
startPoint = null;
endPoint = null;
}

Bitmap/raster software makes use of two memory buffers: one is the current "persisted" canvas that contains the pixels that the user has modified explicitly, the second is the framebuffer on the graphics card which is used to display the canvas on-screen.
Making the bitmap document appear on-screen is done by simply copying the raw bytes of the in-memory bitmap document to the framebuffer (if the framebuffer has a different byte-format or color-depth than the in-memory bitmap then you need to perform a conversion. GDI can do this for you if necessary, but let's just assume everything is 32-bit ARGB).
In WinForms, the framebuffer is exposed by the Graphics argument passed into your Control.OnPaint override.
You can implement these "preview" effects using one of two approaches:
Modern
The first approach is used today, and has been for the past 17 years or so (since Windows 95). The in-memory bitmap is copied to the framebuffer whenever the screen needs to be updated, such as a single mouse movement (of even 1px). The preview effect (such as the line the user would be painting once they release the mouse button) is then drawn on-top. The process is repeated as soon as the user's mouse moves again, so to update the preview.
You'd have something like this:
public class PaintingCanvas : Control {
private Bitmap _canvas = new Bitmap();
private Boolean _inOp; // are we in a mouse operation?
private Point _opStart; // where the current mouse operation started
private Point _opEnd; // where it ends
public override void OnPaint(PaintEventArgs e) {
Graphics g = e.Graphics;
g.DrawImage( _canvas ); // draw the current state
if( _inOp ) {
// assuming the only operation is to draw a line
g.DrawLine( _opStart, _opEnd );
}
}
protected override OnMouseDown(Point p) {
_inOp = true;
_opStart = _opEnd = p;
}
protected override OnMouseMove(Point p) {
_opEnd = p;
this.Invalidate(); // trigger repainting
}
protected override OnMouseUp(Point p) {
using( Graphics g = Graphics.FromImage( _bitmap ) ) {
g.DrawLine( _opStart, _opEnd ); // a permanent line
}
_inOp = false;
}
}
1980s Flashblack
In ye olden days (think: 1980s), copying the bitmap from memory to the framebuffer was slow, so a surprisingly good hack was using XOR painting. The program assumes ownership of the framebuffer (so no overlapping windows would cause it need to be copied from memory). The preview line is drawn by performing an XOR of all of the pixels that the line would cover. This is fast because XORing the pixels means that their original colour can be restored without needing to recopy the pixels from memory. This trick was used in computers for many kinds of selection, highlight, or preview effect until recently.
highlightOrPreviewColor = originalPixelColor XOR (2^bpp - 1)
originalPixelColor = highlightOrPreviewColor XOR (2^bpp - 1)

MediaLibrary.SavePicture fails when fed with a used stream

I need to save the photo that was captured with the CameraCaptureTask, in the Media Library of the phone, and at the same time I want to show this photo in an Image control. The thing is that I first try to rotate the image by accessing its Exif data, so as it appears with the correct orientation in the Image control.
private void cameraTask_Completed(object sender, PhotoResult e)
{
if (e.TaskResult == TaskResult.OK)
{
int angle = GetAngleFromExif(e.ChosenPhoto);
WriteableBitmap currentImage = DecodeImage(e.ChosenPhoto, angle);
photoImage.Source = currentImage;
MediaLibrary medialibrary = new MediaLibrary();
medialibrary.SavePicture("test.jpg", e.ChosenPhoto);
}
}
The code crashes in the last line, with the error:
Value does not fall within the expected range.
What is possibly going wrong here?
Thank you in advance.

Before calling the SavePicture method, you should set the Stream back at the beginning, like this:
e.ChosenPhoto.Seek(0, System.IO.SeekOrigin.Begin);

How can I get a EmguCV ImageBox image?

I'm trying to select a pixel color of the frames of my webcam. So I capture the frames an show then in a ImageBox without any problem. But when I try to access the image stored on ImageBox when I double click on the ImageBox I get a CvException. The exception pop when I try to get the pixel of the image.
OpenCV: unrecognized or unsupported array type
This is how I capture the frames:
// On Form Load
Application.Idle += ProcessFrame;
private void ProcessFrame(object sender, EventArgs arg)
{
if (cap != null)
{
using (Image<Bgr, byte> frame = cap.QueryFrame())
{
if (frame != null)
{
imageFrame = frame;
imageBoxFrame.Image = imageFrame;
Bgr color = imageFrame[50, 100];
}
}
}
}
And in DoubleClick Event:
private void imageBoxFrame_MouseDoubleClick(object sender, MouseEventArgs e)
{
if (treeViewObjects.SelectedNode is ColorNode && !isTracking)
{
if (imageFrame == null)
return;
Emgu.CV.UI.ImageBox imageBox = (Emgu.CV.UI.ImageBox)sender;
Image<Bgr, byte> image = (Image<Bgr, byte>)imageBox.Image;
Bgr color = image[e.X, e.Y]; // This line causes the Exception
}
}
Apparently the image is not null.
What I doing wrong? Maybe something with thread things?

(Question answered by the OP and Answer posting requested. See Question with no answers, but issue solved in the comments (or extended in chat) )
The OP wrote:
I solved it.
I just need to Clone the image because the using statement erase image data. So, on ProcessEvent I just need to clone the frame to imageFrame.
imageFrame = frame.Clone();
And have another problem. The correct way to access the pixel data is by [Y,X] not [X,Y].
Bgr color = image[e.Y, e.X];