Develop Reference

Picturebox from AForge FFMPEG empty - C#/WinForms

I've done a ton of research and looked at a lot of questions here but can't seem to find anything to help me. I should preface I'm very new to C#, Windows Forms, and SO! I'm a 1st year CompSci student coming from C++ experimenting with my own projects for the summer. I'm trying to display a series of bitmaps from a .avi using the AForge.Video.FFMPEG video file reader.
It seems to be finding the file, getting its' data (console prints dimensions, framerate, and codec) and creating the picturebox, but the picturebox comes up blank/empty. I get the bitmap from the frames of a .avi:
From AForge example code here
Then I'm trying to display it with a picture box:
From MS example code here as well
And here's my code. Essentially a combination of the two:
public class Simple : Form
{
Bitmap videoFrame;
public Simple()
{
try
{
// create instance of video reader
VideoFileReader reader = new VideoFileReader();
// open video file
reader.Open(#"C:\Users\User\Desktop\ScanTest3.AVI");
// check some of its attributes
Console.WriteLine("width: " + reader.Width);
Console.WriteLine("height: " + reader.Height);
Console.WriteLine("fps: " + reader.FrameRate);
Console.WriteLine("codec: " + reader.CodecName);
PictureBox pictureBox1 = new PictureBox();
// read 100 video frames out of it
for (int i = 0; i < 100; i++)
{
videoFrame = reader.ReadVideoFrame();
pictureBox1.SizeMode = PictureBoxSizeMode.StretchImage;
pictureBox1.ClientSize = new Size(videoFrame.Width, videoFrame.Height);
pictureBox1.Image = videoFrame;
// dispose the frame when it is no longer required
videoFrame.Dispose();
}
reader.Close();
}
catch
{
Console.WriteLine("Nope");
}
}
}
class MApplication
{
public static void Main()
{
Application.Run(new Simple());
}
}
So that's it pretty much. Just a blank picture box coming up, when it should have the first frame of the video, even though no exception caught (though I'm pretty confident I'm using the try/catch very poorly), and the console printing the correct data for the file:
width: 720
height: 480
fps: 29
codec: dvvideo
[swscaler # 05E10060] Warning: data is not aligned! This can lead to a speedloss
Though if anyone could tell me what that warning means, that would be great as well, but I'm mainly just lost as to why there's no picture printing to the screen.
Thanks!

Found This post, where he uses timers to update, and it works perfectly. I'm guessing pictureBox only really works in a function maybe, not sure. Still learning. Thanks!

Insufficient buffer size using WriteableBitmap?

I am modifying the ColorBasic Kinect example in order to display an image overlaid to the video stream. So what I've done is to load an image with transparent background (now a GIF but it may change), and write to the displayed bitmap.
The error I'm getting is that the buffer I'm writing to is too small.
I cannot see what the actual error is (I'm a complete newbie in XAML/C#/Kinect), but the WriteableBitmap is 1920x1080, and the bitmap I want to copy is 200x200, so why am I getting this error? I cannot see how a transparent background could be of any harm, but I am beginning to suspect that...
Note that without the last WritePixels, the code works and I see the webcam's output. My code follows.
The overlay image:
public BitmapImage overlay = new BitmapImage(new Uri("C:\\users\\user\\desktop\\something.gif"));
The callback function that displays the Kinect's webcam (see the default example ColorBasic) with my very small modifications:
private void Reader_ColorFrameArrived(object sender, ColorFrameArrivedEventArgs e)
{
// ColorFrame is IDisposable
using (ColorFrame colorFrame = e.FrameReference.AcquireFrame())
{
if (colorFrame != null)
{
FrameDescription colorFrameDescription = colorFrame.FrameDescription;
using (KinectBuffer colorBuffer = colorFrame.LockRawImageBuffer())
{
this.colorBitmap.Lock();
// verify data and write the new color frame data to the display bitmap
if ((colorFrameDescription.Width == this.colorBitmap.PixelWidth) && (colorFrameDescription.Height == this.colorBitmap.PixelHeight))
{
colorFrame.CopyConvertedFrameDataToIntPtr(
this.colorBitmap.BackBuffer,
(uint)(colorFrameDescription.Width * colorFrameDescription.Height * 4),
ColorImageFormat.Bgra);
this.colorBitmap.AddDirtyRect(new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight));
}
if(this.overlay != null)
{
// Calculate stride of source
int stride = overlay.PixelWidth * (overlay.Format.BitsPerPixel / 8);
// Create data array to hold source pixel data
byte[] data = new byte[stride * overlay.PixelHeight];
// Copy source image pixels to the data array
overlay.CopyPixels(data, stride, 0);
this.colorBitmap.WritePixels(new Int32Rect(0, 0, overlay.PixelWidth, overlay.PixelHeight), data, stride, 0);
}
this.colorBitmap.Unlock();
}
}
}
}

Your overlay.Format.BitsPerPixel / 8 will be 1 (because it's a gif), but you're trying to copy it to something that is not a gif, probably BGRA (32 bit). Thus you got a huge difference in size (4x).
.WritePixels should take in the stride value of the destination buffer, but you past it the stride value of the overlay (this can cause weird problems as well).
And finally, even if it went 100% smooth your overlay will not actually "overlay" anything, it will replace -- since I don't see any alpha bending math in your code.
Switch your .gif to a .png (32bit) and see if that helps.
Also, if you're looking for an AlphaBltMerge type code: I wrote the entire thing here.. it's very easy to understand.
Merge 2 - 32bit Images with Alpha Channels

How can we capture and save in local disk with Aforge video?

I want to save video which capturing from webcam into local disk. I wrote code it shows webcam but It can't save into local disk. The error is Failed creating compressed stream.. What should I do in here?
writer = new AVIWriter("wmv3");
writer.FrameRate = 30;
writer.Open("video.avi", Convert.ToInt32(640), Convert.ToInt32(480)); // ERROR İS HERE **Failed creating compressed stream.**
//Create NewFrame event handler
//(This one triggers every time a new frame/image is captured
videoSource.NewFrame += new AForge.Video.NewFrameEventHandler(videoSource_NewFrame);
//Start recording
videoSource.Start();
}
}
void videoSource_NewFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
{
//Cast the frame as Bitmap object and don't forget to use ".Clone()" otherwise
//you'll probably get access violation exceptions
pictureBoxVideo.BackgroundImage = (Bitmap)eventArgs.Frame.Clone();
writer.AddFrame((Bitmap)eventArgs.Frame.Clone());
}

Have you ever consider the size of the stream of your webcam? I have the same problem, too. I know you set your video size into 640 and 480, but the video stream size which comes from your webcam(I guess) would be never the same. I also guess you set your container such as picturebox or imagebox into 640 and 480, but that doesn't mean the video stream would be the same. I use savedialog to check the video stream that comes out of my webcam, and guess what? The size would be (648, 486). Who would ever set such a strange number set? But I set my code into this :
writer.Open("video.avi", Convert.ToInt32(648), Convert.ToInt32(486));
And it works fine!
I do not know the rest of your code is correct or not, but I'm sure my bug is in the set of size :)

Four years later I'm having this same issue. After hours I figured out if I didn't specify wmv3 for the codec and just left it blank writer = new AVIWriter(); then everything worked.
AVIWriter write = new AVIWriter();
write.Open("newTestVideo.avi", Convert.ToInt32(320), Convert.ToInt32(240));
Bitmap bit = new Bitmap(320, 240);
for (int tt = 0; tt < 240; tt++) {
bit.SetPixel(tt, tt, System.Drawing.Color.FromArgb((int)(UnityEngine.Random.value * 255f), (int)(UnityEngine.Random.value * 255f), (int)(UnityEngine.Random.value * 255f)));
write.AddFrame(bit);
}
write.Close();

Emgucv Convert<Hsv, Byte>() image

I am having a problem with EmguCV. I used a demo application, and edited it to my needs.
It involves the following function:
public override Image<Gray, byte> DetectSkin(Image<Bgr, byte> Img, IColor min, IColor max)
{
Image<Hsv, Byte> currentHsvFrame = Img.Convert<Hsv, Byte>();
Image<Gray, byte> skin = new Image<Gray, byte>(Img.Width, Img.Height);
skin = currentHsvFrame.InRange((Hsv)min,(Hsv)max);
return skin;
}
In the demo application, the Image comes from a video. The frame is capured from the video like this:
Image<Bgr, Byte> currentFrame;
grabber = new Emgu.CV.Capture(#".\..\..\..\M2U00253.MPG");
grabber.QueryFrame();
currentFrame = grabber.QueryFrame();
In my application, the Image comes from a microsoft kinect stream.
I use the following function:
private void SensorColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
{
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (colorFrame != null)
{
// Copy the pixel data from the image to a temporary array
colorFrame.CopyPixelDataTo(this.colorPixels);
// Write the pixel data into our bitmap
this.colorBitmap.WritePixels(
new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
this.colorPixels,
this.colorBitmap.PixelWidth * sizeof(int),
0);
Bitmap b = BitmapFromWriteableBitmap(this.colorBitmap);
currentFrame = new Image<Bgr, byte>(b);
currentFrameCopy = currentFrame.Copy();
skinDetector = new YCrCbSkinDetector();
Image<Gray, Byte> skin = skinDetector.DetectSkin(currentFrame, YCrCb_min, YCrCb_max);
}
}
}
private static System.Drawing.Bitmap BitmapFromWriteableBitmap(WriteableBitmap writeBmp)
{
System.Drawing.Bitmap bmp;
using (System.IO.MemoryStream outStream = new System.IO.MemoryStream())
{
BitmapEncoder enc = new BmpBitmapEncoder();
enc.Frames.Add(BitmapFrame.Create((BitmapSource)writeBmp));
enc.Save(outStream);
bmp = new System.Drawing.Bitmap(outStream);
}
return bmp;
}
Now, the demo application works, and mine doesn't. Mine gives the following exception:
And, the image here, contains the following:
I really don't understand this exception. And, now, when I run the demo, working aplication, the image, contains:
Which is, in my eyes, exactly the same. I really don't understand this. Help is very welcome!

To make things easier I've uploaded a working WPF solution for you to the code reference sourceforge page I've been building:
http://sourceforge.net/projects/emguexample/files/Capture/Kinect_SkinDetector_WPF.zip/download
https://sourceforge.net/projects/emguexample/files/Capture/
This was designed and tested using EMGU x64 2.42 so in the Lib folder of the project you will find the referenced dlls. If you are using a different version you will need to delete the current references and replace them with the version you're using.
Secondly the project is design like all projects from the code reference library to be built from the Emgu.CV.Example folder into the ..\EMGU 2.X.X.X\bin.. global bin directory where the opencv compiled libraries are within a folder either x86 or x64.
If you struggle to get the code working I can provide all components but I hate redistributing all the opencv files that you already have so let me know if you want this.
You will need to resize the Mainwindow manually to display both images as I didn't spend to much time playing with layout.
So the code...
In the form initialisation method I check for the kinect sensor and set up the eventhandlers for the frames ready. I have left the original threshold values and skinDetector type although I don't use the EMGU version I just forgot to remove it. You will need to play with the threshold values and so on.
//// Look through all sensors and start the first connected one.
//// This requires that a Kinect is connected at the time of app startup.
//// To make your app robust against plug/unplug,
//// it is recommended to use KinectSensorChooser provided in Microsoft.Kinect.Toolkit (See components in Toolkit Browser).
foreach (var potentialSensor in KinectSensor.KinectSensors)
{
if (potentialSensor.Status == KinectStatus.Connected)
{
this.KS = potentialSensor;
break;
}
}
//If we have a Kinect Sensor we will set it up
if (null != KS)
{
// Turn on the color stream to receive color frames
KS.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
//Turn on the depth stream to recieve depth frames
KS.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);
//Start the Streaming process
KS.Start();
//Create a link to a callback to deal with the frames
KS.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(KS_AllFramesReady);
//We set up a thread to process the image/disparty map from the kinect
//Why? The kinect AllFramesReady has a timeout if it has not finished the streams will simply stop
KinectBuffer = new Thread(ProcessBuffer);
hsv_min = new Hsv(0, 45, 0);
hsv_max = new Hsv(20, 255, 255);
YCrCb_min = new Ycc(0, 131, 80);
YCrCb_max = new Ycc(255, 185, 135);
detector = new AdaptiveSkinDetector(1, AdaptiveSkinDetector.MorphingMethod.NONE);
skinDetector = new YCrCbSkinDetector();
}
I always play with the kinect data in a new thread for speed but you may want to advanced this to a Background worker if you plan to do any more heavy processing so it is better managed.
The thread calls the ProcessBuffer() method you can ignore all the commented code as this is the remanence of the code used to display the depth image. Again I'm using the Marshall copy method to keep things fast but the thing to look for is the Dispatcher.BeginInvoke in WPF that allows the images to be displayed from the Kinect thread. This is required as I'm not processing on the main thread.
//This takes the byte[] array from the kinect and makes a bitmap from the colour data for us
byte[] pixeldata = new byte[CF.PixelDataLength];
CF.CopyPixelDataTo(pixeldata);
System.Drawing.Bitmap bmap = new System.Drawing.Bitmap(CF.Width, CF.Height, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
BitmapData bmapdata = bmap.LockBits(new System.Drawing.Rectangle(0, 0, CF.Width, CF.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(pixeldata, 0, ptr, CF.PixelDataLength);
bmap.UnlockBits(bmapdata);
//display our colour frame
currentFrame = new Image<Bgr, Byte>(bmap);
Image<Gray, Byte> skin2 = skinDetector.DetectSkin(currentFrame, YCrCb_min, YCrCb_max);
ExtractContourAndHull(skin2);
DrawAndComputeFingersNum();
//Display our images using WPF Dispatcher Invoke as this is a sub thread.
Dispatcher.BeginInvoke((Action)(() =>
{
ColorImage.Source = BitmapSourceConvert.ToBitmapSource(currentFrame);
}), System.Windows.Threading.DispatcherPriority.Render, null);
Dispatcher.BeginInvoke((Action)(() =>
{
SkinImage.Source = BitmapSourceConvert.ToBitmapSource(skin2);
}), System.Windows.Threading.DispatcherPriority.Render, null);
I hope this helps I will at some point neaten up the code I uploaded,
Cheers

Recording RGB stream from the kinect sensor

I'm doing a WPF application and one of the functions is to record video (Only RGB stream) from Kinect sensor (using Aforge and SDK 1.5).
In my application, i have a button that when clicked, it should save the video stream into an avi file.
I've added the references and I copied all the .dll files into my projects folder (as was explained on other forums) but for some reason I receive this error:
{"Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information.":null}
So inside private void button4_Click(object sender, RoutedEventArgs e) is the following code:
int width = 640;
int height = 480;
// create instance of video writer
VideoFileWriter writer = new VideoFileWriter();
// create new video file
writer.Open("test.avi", width, height, 25, VideoCodec.MPEG4);
// create a bitmap to save into the video file
Bitmap image = new Bitmap (width, height, DrawingColor.PixelFormat.Format24bppRgb);
for (int i = 0; i < 1000; i++)
{
image.SetPixel(i % width, i % height, Color.Red);
writer.WriteVideoFrame(image);
}
writer.Close();
}
I will really appreciate your help and also im flexible with the way to record RGB stream(if you recommend another way), as long a its not complicated because im new with C#

The reason the video is red is because you are turning it red with
for (int i = 0; i < 1000; i++)
{
image.SetPixel(i % width, i % height, Color.Red);
writer.WriteVideoFrame(image);
}
What you should do is convert the BitmapSource/WritableBitmap* (assuming you are displaying Kinect's data with a BitmapSource or WritableBitmap Then you can just add that bitmap to your Video frame. Hope this helps!
**If you are using a WritableBitmap, convert it to a BitmapImage, then convert that to a Bitmap*