Develop Reference

C# .Net Emgu.CV is inconsistently transferring bytes to itself

I'm using Emgu.CV to templateMatch and to save Images.
Unfortunetly I have ran into an issue that I have no been able to solve for a weeks.
Problem is that i serialize byte array and size from original Image to json file, and whenever i try to convert it back sometimes the image is distorted.
I have already tried skipping over serializing procces and it still became distorted.
Here is code of converting procces:
Image<Bgr565, byte> screenCrop = SnipMaker.takeSnip();//method creates screenshot at this point when i display the images they are 100% correct
byte[] data = screenCrop.Bytes;//I would get normaly all this from json file(in this case im skipping over it)
Mat mat = new Mat(screenCrop.Rows, screenCrop.Cols, screenCrop.Mat.Depth, asset.NumberOfChannels);
Marshal.Copy(data, 0, mat.DataPointer, screenCrop.asset.Cols * screenCrop.asset.Rows * asset.NumberOfChannels);
Image<Bgr565, byte> img = mat.ToImage<Bgr565, byte>();//This image is suddenly distorted
Problem is that this results depending on "I'm not sure what" is either prefecly good image or skwed one:
normal result
same code different result
Its almost like its sometimes 1 pixel behind but only thing that is changing is size and dimentions of screen shots.
I have tried dirrect ways like
Image<Bgr, byte> img = new Image<Bgr, byte>(width, height);
img.Bytes = data;//data is byte array that i got from file
This also gives sometimes correct picture but other times it throws an exeption (out of range exception in marshal.cs when trying to copy bytes from data to img)
only thing that i suspect at this point is that im doing something wrong whenever im taking screenshot but im not sure what:
public static Image<Bgr565, byte> Snip()
{
int screenWidth = (int)System.Windows.SystemParameters.PrimaryScreenWidth;
int screenHeight = (int)System.Windows.SystemParameters.PrimaryScreenHeight;
using (Bitmap bmp = new Bitmap(screenWidth, screenHeight))
{
using (Graphics gr = Graphics.FromImage(bmp))
gr.CopyFromScreen(0, 0, 0, 0, bmp.Size);
using (var snipper = new SnippingTool(bmp))
{
if (snipper.ShowDialog() == true)
{
Bitmap bitmapImage = new Bitmap(snipper.Image);
Rectangle rectangle = new Rectangle(0, 0, bitmapImage.Width, bitmapImage.Height);//System.Drawing
BitmapData bmpData = bitmapImage.LockBits(rectangle, ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format24bppRgb);//System.Drawing.Imaging
Image<Bgr565, byte> outputImage = new Image<Bgr565, byte>(bitmapImage.Width, bitmapImage.Height, bmpData.Stride, bmpData.Scan0);
bitmapImage.Dispose();
snipper.Close();
return outputImage;
}
}
return null;
}
}
So far I have not been able to solve this and knowing my luck noone will proppably anwser me here. But please could someone help me with this?
Thank you in advance

So thank you to everyones help.
The issue was indeed in the screenshot script. I've used incorrect combination of
pixel formats which resulted in inconsistent bit transfer.
But because the step property in Image<bgr,byte>.Mat was calculated based on the width of the image (Emgucv SC):
step = sizeof(byte) * s.Width * channels;
It caused that some of the images looked normal and other didn't.(speculation based on observation)
Fix:
change all Image<Bgr, byte> to Image<Bgra, byte>
to make it 32bit and then change:
BitmapData bmpData = bitmapImage.LockBits(rectangle, ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
to:
BitmapData bmpData = bitmapImage.LockBits(rectangle, ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
Hope this will help someone in the future. : )

C# save image to file system, error in gdi+

I have a C# WPF application which cuts a image to the size I need it. The WPF window is used to put in a user ID to save the image into a database. When I test my application it works sometimes and sometimes i get a error in gdi+ on the line I save the image to the file system.
Here is my code:
public static void CutImage(Image image)
{
//create new image
Bitmap source = new Bitmap(image);
//cut image
Bitmap cuttedImage = source.Clone(new System.Drawing.Rectangle(250, 0, 5550, 4000), source.PixelFormat);
//copy bitmap to avoid "general error in gdi+"
Bitmap copyImage = new Bitmap(cuttedImage.Width, cuttedImage.Height, PixelFormat.Format24bppRgb);
Graphics g = Graphics.FromImage(copyImage);
g.DrawImageUnscaled(cuttedImage, 0, 0);
//dispose graphics object
g.Dispose();
//dispose image
cuttedImage.Dispose();
//save image to filesystem
copyImage.Save(#"path\tmp.jpg", ImageFormat.Jpeg);
}
//get image
Image i = Image.FromFile(path\image.jpg);
//cut image
CutImage(i);
I searched for a solution and somebody said I have to create a copy of the image I got from Image.FromFile(). But the error still happens sometimes. I tried it a few times and it seems to be random when it happens. The error is always on the Image.Save() line.
Does somebody know how to solve this problem or is there a alternative to Image.Save()?
Thanks for your help!

You're creating so many bitmaps and you're not disposing them. Every single IDisposable must be explicitly disposed when you're finished with them.
Try this and see if the error goes away:
public static void CutImage(Image image)
{
using (Bitmap source = new Bitmap(image))
{
using (Bitmap cuttedImage = source.Clone(new System.Drawing.Rectangle(250, 0, 5550, 4000), source.PixelFormat))
{
using (Bitmap copyImage = new Bitmap(cuttedImage.Width, cuttedImage.Height, System.Drawing.Imaging.PixelFormat.Format24bppRgb))
{
using (Graphics g = Graphics.FromImage(copyImage))
{
g.DrawImageUnscaled(cuttedImage, 0, 0);
copyImage.Save(#"path\tmp.jpg", System.Drawing.Imaging.ImageFormat.Jpeg);
}
}
}
}
}

Taking a screenshot using Graphics.CopyFromScreen with 150% scaling

I'm trying to create a screenshot/bitmap of my screen. I wrote this function:
public static Bitmap CreateScreenshot(Rectangle bounds)
{
var bmpScreenshot = new Bitmap(bounds.Width, bounds.Height,
PixelFormat.Format32bppArgb);
var gfxScreenshot = Graphics.FromImage(bmpScreenshot);
gfxScreenshot.CopyFromScreen(bounds.X, bounds.Y,
0, 0,
new Size(bounds.Size.Width, bounds.Size.Height),
CopyPixelOperation.SourceCopy);
return bmpScreenshot;
}
This function is being called in my overlay form that should draw the bitmap onto itself. I'm currently using GDI+ for the whole process.
private void ScreenshotOverlay_Load(object sender, EventArgs e)
{
foreach (Screen screen in Screen.AllScreens)
Size += screen.Bounds.Size;
Location = Screen.PrimaryScreen.Bounds.Location;
_screenshot = BitmapHelper.CreateScreenshot(new Rectangle(new Point(0, 0), Size));
Invalidate(); // The screenshot/bitmap is drawn here
}
Yep, I dispose the bitmap later, so don't worry. ;)
On my laptop and desktop computer this works fine. I've tested this with different resolutions and the calculations are correct. I can see an image of the screen on the form.
The problem starts with the Surface 3. All elements are being scaled by a factor of 1.5 (150%). This consequently means that the DPI changes. If I try to take a screenshot there, it does only capture like the upper-left part of the screen but not the whole one.
I've made my way through Google and StackOverflow and tried out different things:
Get the DPI, divide it by 96 and multiply the size components (X and Y) of the screen with this factor.
Add an entry to application.manifest to make the application DPI-aware.
The first way did not bring the desired result. The second one did, but the whole application would have to be adjusted then and this is quite complicated in Windows Forms.
Now my question would be: Is there any way to capture a screenshot of the whole screen, even if it is has a scalation factor higher than 1 (higher DPI)?
There must be a way to do this in order to make it working everywhere.
But at this point I had no real search results that could help me.
Thanks in advance.

Try this, which is found within SharpAVI's library. It works well on devices regardless of resolution scale. And I have tested it on Surface 3 at 150%.
System.Windows.Media.Matrix toDevice;
using (var source = new HwndSource(new HwndSourceParameters()))
{
toDevice = source.CompositionTarget.TransformToDevice;
}
screenWidth = (int)Math.Round(SystemParameters.PrimaryScreenWidth * toDevice.M11);
screenHeight = (int)Math.Round(SystemParameters.PrimaryScreenHeight * toDevice.M22);
SharpAVI can be found here: https://github.com/baSSiLL/SharpAvi It is for videos but uses a similar copyFromScreen method when getting each frame:
graphics.CopyFromScreen(0, 0, 0, 0, new System.Drawing.Size(screenWidth, screenHeight));

Before taking your screen shot, you can make the process DPI aware:
[System.Runtime.InteropServices.DllImport("user32.dll")]
public static extern bool SetProcessDPIAware();
private static Bitmap Screenshot()
{
SetProcessDPIAware();
var screen = System.Windows.Forms.Screen.PrimaryScreen;
var rect = screen.Bounds;
var size = rect.Size;
Bitmap bmpScreenshot = new Bitmap(size.Width, size.Height);
Graphics g = Graphics.FromImage(bmpScreenshot);
g.CopyFromScreen(0, 0, 0, 0, size);
return bmpScreenshot;
}

Emgucv Convert<Hsv, Byte>() image

I am having a problem with EmguCV. I used a demo application, and edited it to my needs.
It involves the following function:
public override Image<Gray, byte> DetectSkin(Image<Bgr, byte> Img, IColor min, IColor max)
{
Image<Hsv, Byte> currentHsvFrame = Img.Convert<Hsv, Byte>();
Image<Gray, byte> skin = new Image<Gray, byte>(Img.Width, Img.Height);
skin = currentHsvFrame.InRange((Hsv)min,(Hsv)max);
return skin;
}
In the demo application, the Image comes from a video. The frame is capured from the video like this:
Image<Bgr, Byte> currentFrame;
grabber = new Emgu.CV.Capture(#".\..\..\..\M2U00253.MPG");
grabber.QueryFrame();
currentFrame = grabber.QueryFrame();
In my application, the Image comes from a microsoft kinect stream.
I use the following function:
private void SensorColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
{
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (colorFrame != null)
{
// Copy the pixel data from the image to a temporary array
colorFrame.CopyPixelDataTo(this.colorPixels);
// Write the pixel data into our bitmap
this.colorBitmap.WritePixels(
new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
this.colorPixels,
this.colorBitmap.PixelWidth * sizeof(int),
0);
Bitmap b = BitmapFromWriteableBitmap(this.colorBitmap);
currentFrame = new Image<Bgr, byte>(b);
currentFrameCopy = currentFrame.Copy();
skinDetector = new YCrCbSkinDetector();
Image<Gray, Byte> skin = skinDetector.DetectSkin(currentFrame, YCrCb_min, YCrCb_max);
}
}
}
private static System.Drawing.Bitmap BitmapFromWriteableBitmap(WriteableBitmap writeBmp)
{
System.Drawing.Bitmap bmp;
using (System.IO.MemoryStream outStream = new System.IO.MemoryStream())
{
BitmapEncoder enc = new BmpBitmapEncoder();
enc.Frames.Add(BitmapFrame.Create((BitmapSource)writeBmp));
enc.Save(outStream);
bmp = new System.Drawing.Bitmap(outStream);
}
return bmp;
}
Now, the demo application works, and mine doesn't. Mine gives the following exception:
And, the image here, contains the following:
I really don't understand this exception. And, now, when I run the demo, working aplication, the image, contains:
Which is, in my eyes, exactly the same. I really don't understand this. Help is very welcome!

To make things easier I've uploaded a working WPF solution for you to the code reference sourceforge page I've been building:
http://sourceforge.net/projects/emguexample/files/Capture/Kinect_SkinDetector_WPF.zip/download
https://sourceforge.net/projects/emguexample/files/Capture/
This was designed and tested using EMGU x64 2.42 so in the Lib folder of the project you will find the referenced dlls. If you are using a different version you will need to delete the current references and replace them with the version you're using.
Secondly the project is design like all projects from the code reference library to be built from the Emgu.CV.Example folder into the ..\EMGU 2.X.X.X\bin.. global bin directory where the opencv compiled libraries are within a folder either x86 or x64.
If you struggle to get the code working I can provide all components but I hate redistributing all the opencv files that you already have so let me know if you want this.
You will need to resize the Mainwindow manually to display both images as I didn't spend to much time playing with layout.
So the code...
In the form initialisation method I check for the kinect sensor and set up the eventhandlers for the frames ready. I have left the original threshold values and skinDetector type although I don't use the EMGU version I just forgot to remove it. You will need to play with the threshold values and so on.
//// Look through all sensors and start the first connected one.
//// This requires that a Kinect is connected at the time of app startup.
//// To make your app robust against plug/unplug,
//// it is recommended to use KinectSensorChooser provided in Microsoft.Kinect.Toolkit (See components in Toolkit Browser).
foreach (var potentialSensor in KinectSensor.KinectSensors)
{
if (potentialSensor.Status == KinectStatus.Connected)
{
this.KS = potentialSensor;
break;
}
}
//If we have a Kinect Sensor we will set it up
if (null != KS)
{
// Turn on the color stream to receive color frames
KS.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
//Turn on the depth stream to recieve depth frames
KS.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);
//Start the Streaming process
KS.Start();
//Create a link to a callback to deal with the frames
KS.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(KS_AllFramesReady);
//We set up a thread to process the image/disparty map from the kinect
//Why? The kinect AllFramesReady has a timeout if it has not finished the streams will simply stop
KinectBuffer = new Thread(ProcessBuffer);
hsv_min = new Hsv(0, 45, 0);
hsv_max = new Hsv(20, 255, 255);
YCrCb_min = new Ycc(0, 131, 80);
YCrCb_max = new Ycc(255, 185, 135);
detector = new AdaptiveSkinDetector(1, AdaptiveSkinDetector.MorphingMethod.NONE);
skinDetector = new YCrCbSkinDetector();
}
I always play with the kinect data in a new thread for speed but you may want to advanced this to a Background worker if you plan to do any more heavy processing so it is better managed.
The thread calls the ProcessBuffer() method you can ignore all the commented code as this is the remanence of the code used to display the depth image. Again I'm using the Marshall copy method to keep things fast but the thing to look for is the Dispatcher.BeginInvoke in WPF that allows the images to be displayed from the Kinect thread. This is required as I'm not processing on the main thread.
//This takes the byte[] array from the kinect and makes a bitmap from the colour data for us
byte[] pixeldata = new byte[CF.PixelDataLength];
CF.CopyPixelDataTo(pixeldata);
System.Drawing.Bitmap bmap = new System.Drawing.Bitmap(CF.Width, CF.Height, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
BitmapData bmapdata = bmap.LockBits(new System.Drawing.Rectangle(0, 0, CF.Width, CF.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(pixeldata, 0, ptr, CF.PixelDataLength);
bmap.UnlockBits(bmapdata);
//display our colour frame
currentFrame = new Image<Bgr, Byte>(bmap);
Image<Gray, Byte> skin2 = skinDetector.DetectSkin(currentFrame, YCrCb_min, YCrCb_max);
ExtractContourAndHull(skin2);
DrawAndComputeFingersNum();
//Display our images using WPF Dispatcher Invoke as this is a sub thread.
Dispatcher.BeginInvoke((Action)(() =>
{
ColorImage.Source = BitmapSourceConvert.ToBitmapSource(currentFrame);
}), System.Windows.Threading.DispatcherPriority.Render, null);
Dispatcher.BeginInvoke((Action)(() =>
{
SkinImage.Source = BitmapSourceConvert.ToBitmapSource(skin2);
}), System.Windows.Threading.DispatcherPriority.Render, null);
I hope this helps I will at some point neaten up the code I uploaded,
Cheers

Applying BackwardQuadrilateralTransformation to defined GRAFT points

I have a small piece of C# code that uses a Kinect to detect up to 4 glyphs and draws and polygon between them on a canvas, as seen here:
I've tried to follow along to this in order to implement 2D augmented reality and project an image within the created polygon. I've read in a source image and tried to apply the BackwardQuadrilateralTransformation to it but can't seem to display the transformed image. I am probably using the wrong method but I have tried to convert the image and paint it onto a canvas with no luck. I'm not sure if I'm just massively misunderstanding the method and maybe it isn't possible, any help would be greatly appreciated. I can supply more sample code if required.
private void GlyphBackQuad(List<IntPoint> quadpoints)
{
Bitmap srcImage = new Bitmap( // my sample image filepath );
UnmanagedImage sourceImage = UnmanagedImage.FromManagedImage(srcImage);
BackwardQuadrilateralTransformation filter = new BackwardQuadrilateralTransformation(sourceImage, quadpoints);
filter.Apply(sourceImage);
Bitmap bmp = sourceImage.ToManagedImage();
ImageBrush ib = new ImageBrush();
ib.ImageSource = ConvertDrawingImage2MediaImageSource(bmp);
PolyCanvas.Background = ib;
}
After a further play around with the code I think I have developed a partial solution, it still needs some work but hopefully this is more helpful for people to read/debug.
private void GlyphBackQuad(List<IntPoint> quadpoints, Bitmap bmp)
{
// Read in bitmap source image and clone it to the same format as destination
Bitmap srcImage = AForge.Imaging.Image.Clone(new Bitmap( // my sample filepath), System.Drawing.Imaging.PixelFormat.Format24bppRgb);
System.Drawing.Imaging.BitmapData bitmapData = bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
// Convert to unmanaged image
UnmanagedImage unmanagedImage = new UnmanagedImage( bitmapData );
// Filter
BackwardQuadrilateralTransformation filter = new BackwardQuadrilateralTransformation(srcImage, quadpoints);
filter.ApplyInPlace(unmanagedImage);
// Convert back to managed image and save
Bitmap managedImage = unmanagedImage.ToManagedImage();
managedImage.Save( // my save filepath, System.Drawing.Imaging.ImageFormat.Png);
}