I'm loading C++ library from my C# code dynamically. I want to find small image inside large one, converting large image to byte[] and small image read from physical path. When I call imdecode then large_img always returns 0 cols and rows.
C#
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate ImageParams GetImageParams(IntPtr dataPtr, int size, string path);
// ...
byte[] largeImgByteArr = this.BitmapToByteArray(bmp);
IntPtr dataPtr = Marshal.AllocHGlobal(largeImgByteArr.Length);
Marshal.Copy(dataPtr, largeImgByteArr, 0, largeImgByteArr.Length);
C++
ImageParams GetImageParams(BYTE* largeImgBuf, int bufLength, const char* smallImgPath)
{
Mat large_img_data(bufLength, 1, CV_32FC1, largeImgBuf);
Mat large_img = imdecode(large_img_data, IMREAD_COLOR);
Mat small_img = imread(smallImgPath, IMREAD_COLOR);
int result_cols = large_img.cols - small_img.cols + 1;
int result_rows = large_img.rows - small_img.rows + 1;
Mat result;
result.create(result_cols, result_rows, CV_32FC1);
matchTemplate(large_img, small_img, result, CV_TM_SQDIFF_NORMED);
normalize(result, result, 0, 1, NORM_MINMAX, -1, Mat());
}
What I'm doing wrong here?
Note: I have checked that image path is correct and byte array not empty.
Edit 1
I changed my code a bit by providing the large image width and height, also got rid of imdecode and changed something like in this post.
ImageParams GetImageParams(BYTE* largeImgBuf, int height, int width, int bufLength, const char* smallImgPath)
{
// Mat large_img = imdecode(large_img_data, IMREAD_COLOR);
Mat large_img = Mat(height, width, CV_8UC3, largeImgBuf);
Mat small_img = imread(templPath, 1);
/// ...
}
Now it returns rows and columns but when call matchTemplate method it throws an exception:
Remember that Bitmap structure in C# uses data padding (stride value as multiplicity of 4) whrereas OpenCV may not. Try creating Mat object directly from byte array but adjust step (stride) value as it just assigns data without any ownership or reallocation.
EDIT
Here's an example how create OpenCV Mat object from Bitmap. Data from bitmap is not copied, but only assigned. PixelFormat and OpenCV mat type must have corresponding single element size in bytes.
cv::Mat ImageBridge::cvimage(System::Drawing::Bitmap^ bitmap){
if(bitmap != nullptr){
switch(bitmap->PixelFormat){
case System::Drawing::Imaging::PixelFormat::Format24bppRgb:
return bmp2mat(bitmap, CV_8UC3);
case System::Drawing::Imaging::PixelFormat::Format8bppIndexed:
return bmp2mat(bitmap, CV_8U);
default: return cv::Mat();
}
}
else
return cv::Mat();
}
cv::Mat ImageBridge::bmp2mat(System::Drawing::Bitmap^ bitmap, int image_type){
auto bitmap_data = bitmap->LockBits(
System::Drawing::Rectangle(0, 0, bitmap->Width, bitmap->Height),
System::Drawing::Imaging::ImageLockMode::ReadWrite,
bitmap->PixelFormat);
char* bmpptr = (char*)bitmap_data->Scan0.ToPointer();
cv::Mat image(
cv::Size(bitmap->Width, bitmap->Height),
image_type,
bmpptr,
bitmap_data->Stride);
bitmap->UnlockBits(bitmap_data);
return image;
}
EDIT 2
Conversion in reverse - this time data from Mat image is copied as Bitmap allocates it's own memory.
System::Drawing::Bitmap^ ImageBridge::bitmap(cv::Mat& image){
if(!image.empty() && image.type() == CV_8UC3)
return mat2bmp(image, System::Drawing::Imaging::PixelFormat::Format24bppRgb);
else if(!image.empty() && image.type() == CV_8U)
return mat2bmp(image, System::Drawing::Imaging::PixelFormat::Format8bppIndexed);
else
return nullptr;
}
System::Drawing::Bitmap^ ImageBridge::mat2bmp(cv::Mat& image, System::Drawing::Imaging::PixelFormat pixel_format){
if(image.empty())
return nullptr;
System::Drawing::Bitmap^ bitmap = gcnew System::Drawing::Bitmap(image.cols, image.rows, pixel_format);
auto bitmap_data = bitmap->LockBits(
System::Drawing::Rectangle(0, 0, bitmap->Width, bitmap->Height),
System::Drawing::Imaging::ImageLockMode::ReadWrite,
pixel_format);
char* bmpptr = (char*)bitmap_data->Scan0.ToPointer();
int line_length = (int)image.step;
int bmp_stride = bitmap_data->Stride;
assert(!image.isSubmatrix());
assert(bmp_stride >= 0);
for(int l = 0; l < image.rows; l++){
char* cvptr = (char*)image.ptr(l);
int bmp_line_index = l * bmp_stride;
for(int i = 0; i < line_length; ++i)
bmpptr[bmp_line_index + i] = cvptr[i];
}
bitmap->UnlockBits(bitmap_data);
return bitmap;
}
Or if you have Mat image with step as multiplicity of 4 you can use non-copy version.
System::Drawing::Bitmap^ ImageBridge::bitmap2(cv::Mat& image){
System::Drawing::Bitmap^ bitmap;
assert(!image.isSubmatrix());
if(!image.empty() && image.type() == CV_8UC3){
bitmap = gcnew System::Drawing::Bitmap(
image.cols, image.rows,
3 * image.cols,
System::Drawing::Imaging::PixelFormat::Format24bppRgb,
System::IntPtr(image.data));
}
else if(!image.empty() && image.type() == CV_8U){
bitmap = gcnew System::Drawing::Bitmap(
image.cols, image.rows,
image.cols,
System::Drawing::Imaging::PixelFormat::Format8bppIndexed,
System::IntPtr(image.data));
}
return bitmap;
}
According to the documentation
The function reads an image from the specified buffer in the memory. If the buffer is too short or contains invalid data, the empty matrix/image is returned.
Try checking the error code after the imdecode part
#include <errno.h>
cout << errno;
Related
I need to change my screen capture code to get a pixel array instead of a Bitmap.
I change the code to this:
BitBlt > Image.FromHbitmap(pointer) > LockBits > pixel array
But, I'm checking if it's possible to cut some middle man, and have something like this:
BitBlt > Marshal.Copy > pixel array
Or even:
WinApi method that gets the screen region as a pixel array
So far, I tried to use this code, without success:
public static byte[] CaptureAsArray(Size size, int positionX, int positionY)
{
var hDesk = GetDesktopWindow();
var hSrce = GetWindowDC(hDesk);
var hDest = CreateCompatibleDC(hSrce);
var hBmp = CreateCompatibleBitmap(hSrce, (int)size.Width, (int)size.Height);
var hOldBmp = SelectObject(hDest, hBmp);
try
{
new System.Security.Permissions.UIPermission(System.Security.Permissions.UIPermissionWindow.AllWindows).Demand();
var b = BitBlt(hDest, 0, 0, (int)size.Width, (int)size.Height, hSrce, positionX, positionY, CopyPixelOperation.SourceCopy | CopyPixelOperation.CaptureBlt);
var length = 4 * (int)size.Width * (int)size.Height;
var bytes = new byte[length];
Marshal.Copy(hBmp, bytes, 0, length);
//return b ? Image.FromHbitmap(hBmp) : null;
return bytes;
}
finally
{
SelectObject(hDest, hOldBmp);
DeleteObject(hBmp);
DeleteDC(hDest);
ReleaseDC(hDesk, hSrce);
}
return null;
}
This code gives me an System.AccessViolationException while stepping on Marshal.Copy.
Is there any more efficient way of getting screen pixels as a byte array while using BitBlt or similar screen capture methods?
EDIT:
As found in here and as suggested by CodyGray, I should use
var b = Native.BitBlt(_compatibleDeviceContext, 0, 0, Width, Height, _windowDeviceContext, Left, Top, Native.CopyPixelOperation.SourceCopy | Native.CopyPixelOperation.CaptureBlt);
var bi = new Native.BITMAPINFOHEADER();
bi.biSize = (uint)Marshal.SizeOf(bi);
bi.biBitCount = 32;
bi.biClrUsed = 0;
bi.biClrImportant = 0;
bi.biCompression = 0;
bi.biHeight = Height;
bi.biWidth = Width;
bi.biPlanes = 1;
var data = new byte[4 * Width * Height];
Native.GetDIBits(_windowDeviceContext, _compatibleBitmap, 0, (uint)Height, data, ref bi, Native.DIB_Color_Mode.DIB_RGB_COLORS);
My data array has all the pixels of the screenshot.
Now, I'm going to test if there's any performance improvements or not.
Yeah, you can't just start accessing the raw bits of a BITMAP object through an HBITMAP (as returned by CreateCompatibleBitmap). HBITMAP is just a handle, as the name suggests. It's not a pointer in the classic "C" sense that it points to the beginning of the array. Handles are like indirect pointers.
GetDIBits is the appropriate solution to get the raw, device-independent pixel array from a bitmap that you can iterate through. But you'll still need to use the code you have to get the screen bitmap in the first place. Essentially, you want something like this. Of course, you'll need to translate it into C#, but that shouldn't be difficult, since you already know how to call WinAPI functions.
Note that you do not need to call GetDesktopWindow or GetWindowDC. Just pass NULL as the handle to GetDC; it has the same effect of returning a screen DC, which you can then use to create a compatible bitmap. In general, you should almost never call GetDesktopWindow.
I'm making a program which can capture a small area on screen and will run something if there is any color on image that match the target colors. My program run as the following Sequence:
Get image from a specific area from screen
Save to a folder
using CountPixel to detect any target_color
However, after I click the button5 twice times (not double click), it through an exception at below line :
b.Save(#"C:\Applications\CaptureImage000.jpg", ImageFormat.Jpeg);
Exception :
An unhandled exception of type
'System.Runtime.InteropServices.ExternalException' occurred in
System.Drawing.dll
Additional information: A generic error occurred in GDI+
My questions are :
How can i fix this exception ?
I want to use another method instead of CountPixel() to improve performance, because I just need to detect only one target color to rise event.
Step 2 is troublesome. I wonder if i can skip it and use the other way to call: (#"C:\Applications\CaptureImage000.jpg", ImageFormat.Jpeg) , because using this long string isn't comfortable and result error when im trying to use with GetPixel,... or add it into some "value example" code on internet for improvement purpose.
private int CountPixels(Bitmap bm, Color target_color)
{
// Loop through the pixels.
int matches = 0;
for (int y = 0; y < bm.Height; y++)
{
for (int x = 0; x < bm.Width; x++)
{
if (bm.GetPixel(x, y) == target_color) matches++;
}
}
return matches;
}
private Bitmap CapturedImage(int x, int y)
{
Bitmap b = new Bitmap(XX, YY);
Graphics g = Graphics.FromImage(b);
g.CopyFromScreen(x, y, 0, 0, new Size(XX, YY));
b.Save(#"C:\Applications\CaptureImage000.jpg", ImageFormat.Jpeg);
/* Run 3 line below will lead to question 1 - through exception
Bitmap bm = new Bitmap(#"C:\Applications\CaptureImage000.jpg");
int black_pixels = CountPixels(b, Color.FromArgb(255, 0, 0, 0));
textBox3.Text = black_pixels + " black pixels";
*/
return b;
}
private void button5_Click(object sender, EventArgs e)// Do screen cap
{
Bitmap bmp = null;
bmp = CapturedImage(X0, Y0);
}
[EDIT] Worked on this tonight with OP, made some improvements
This now accounts for endianness of the machine and correctly compares colors by converting them to integers with the Color.ToArgb() function
the below code will work, I have added comments for clarity and given you some options. I wrote the code without an IDE but I am confident it is fine.
In both cases below, just keep the handle to the bitmap, don't need to save and reopen regardless of if you need a copy.
Exception issue and improvements to CapturedImage function
option A (recommended)
Don't save the bitmap, you already have a handle, the graphics object just modified the BMP. Just leave the below code as is for this function and it will work fine without un-commenting one of the other options.
Code and other options:
private Bitmap CapturedImage(Bitmap bm, int x, int y)
{
Bitmap b = new Bitmap(XX, YY);
Graphics g = Graphics.FromImage(b);
g.CopyFromScreen(x, y, 0, 0, new Size(XX, YY));
//option B - If you DO need to keep a copy of the image use PNG and delete the old image
/*
try
{
if(System.IO.File.Exists(#"C:\Applications\CaptureImage.png"))
{
System.IO.File.Delete(#"C:\Applications\CaptureImage.png");
}
b.Save(#"C:\Applications\CaptureImage.png", ImageFormat.Png);
}
catch (System.Exception ex)
{
MessageBox.Show("There was a problem trying to save the image, is the file in open in another program?\r\nError:\r\n\r\n" + ex.Message);
}
*/
//option C - If you DO need to keep a copy of the image AND keep all copies of all images when you click the button use PNG and generate unique filename
/*
int id = 0;
while(System.IO.File.Exists(#"C:\Applications\CaptureImage" + id.ToString().PadLeft('0',4) + ".png"))
{
//increment the id until a unique file name is found
id++;
}
b.Save(#"C:\Applications\CaptureImage" + id.ToString().PadLeft('0',4) + ".png", ImageFormat.Png);
*/
int black_pixels = CountPixels(b, Color.FromArgb(255, 0, 0, 0));
textBox3.Text = black_pixels + " black pixels";
return b;
}
Now for the CountPixels function, you have 3 options but really, you have one really solid option, so I am omitting the others.
This locks the bits in the BMP, uses marshalling to copy the data into an array and scans the array for data, very, very fast, and you will likely not even need to remove the count. If you do STILL want to remove the count, just add "return 1;" right underneath where it increments the matches variable.
Speed issue and improvements to CountPixels function
private int CountPixels(Bitmap bm, Color target_color)
{
int matches = 0;
Bitmap bmp = (Bitmap)bm.Clone();
BitmapData bmpDat = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat);
int size = bmpDat.Stride * bmpDat.Height;
byte[] subPx = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(bmpDat.Scan0, subPx, 0, size);
//change the 4 (ARGB) to a 3 (RGB) if you don't have an alpha channel, this is for 32bpp images
//ternary operator to check endianess of machine and organise pixel colors as A,R,G,B or B,G,R,A (little endian is reversed);
Color temp = BitConverter.IsLittleEndian ? Color.FromArgb(subPx[i + 2], subPx[i + 1], subPx[i]) : Color.FromArgb(subPx[i + 1], subPx[i + 2], subPx[i + 3]);
for (int i = 0; i < size; i += 4 ) //4 bytes per pixel A, R, G, B
{
if(temp.ToArgb() == target_color.ToArgb())
{
matches++;
}
}
System.Runtime.InteropServices.Marshal.Copy(subPx, 0, bmpDat.Scan0, subPx.Length);
bmp.UnlockBits(bmpDat);
return matches;
}
Finally the same function but allowing for a tolerance percent
private int CountPixels(Bitmap bm, Color target_color, float tolerancePercent)
{
int matches = 0;
Bitmap bmp = (Bitmap)bm.Clone();
BitmapData bmpDat = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat);
int size = bmpDat.Stride * bmpDat.Height;
byte[] subPx = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(bmpDat.Scan0, subPx, 0, size);
for (int i = 0; i < size; i += 4 )
{
byte r = BitConverter.IsLittleEndian ? subPx[i+2] : subPx[i+3];
byte g = BitConverter.IsLittleEndian ? subPx[i+1] : subPx[i+2];
byte b = BitConverter.IsLittleEndian ? subPx[i] : subPx[i+1];
float distancePercent = (float)Math.Sqrt(
Math.Abs(target_color.R-r) +
Math.Abs(target_color.G-g) +
Math.Abs(target_color.B-b)
) / 7.65f;
if(distancePercent < tolerancePercent)
{
matches++;
}
}
System.Runtime.InteropServices.Marshal.Copy(subPx, 0, bmpDat.Scan0, subPx.Length);
bmp.UnlockBits(bmpDat);
return matches;
}
I know how to do it in WPF but I have problem for capturing depth in winforms application.
I found some code as below:
private void Kinect_DepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
{
using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
if (depthFrame != null)
{
Bitmap DepthBitmap = new Bitmap(depthFrame.Width, depthFrame.Height, PixelFormat.Format32bppRgb);
if (_depthPixels.Length != depthFrame.PixelDataLength)
{
_depthPixels = new DepthImagePixel[depthFrame.PixelDataLength];
_mappedDepthLocations = new ColorImagePoint[depthFrame.PixelDataLength];
}
//Copy the depth frame data onto the bitmap
var _pixelData = new short[depthFrame.PixelDataLength];
depthFrame.CopyPixelDataTo(_pixelData);
BitmapData bmapdata = DepthBitmap.LockBits(new Rectangle(0, 0, depthFrame.Width,
depthFrame.Height), ImageLockMode.WriteOnly, DepthBitmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(_pixelData, 0, ptr, depthFrame.Width * depthFrame.Height);
DepthBitmap.UnlockBits(bmapdata);
pictureBox2.Image = DepthBitmap;
}
}
}
but this is not giving me the greyScale depth and it's purple. Any improvement or help?
I found the solution myself, by a function to convert the depth frame:
void Kinect_DepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
{
using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
if (depthFrame != null)
{
this.depthFrame32 = new byte[depthFrame.Width * depthFrame.Height * 4];
//Update the image to the new format
this.depthPixelData = new short[depthFrame.PixelDataLength];
depthFrame.CopyPixelDataTo(this.depthPixelData);
byte[] convertedDepthBits = this.ConvertDepthFrame(this.depthPixelData, ((KinectSensor)sender).DepthStream);
Bitmap bmap = new Bitmap(depthFrame.Width, depthFrame.Height, PixelFormat.Format32bppRgb);
BitmapData bmapdata = bmap.LockBits(new Rectangle(0, 0, depthFrame.Width, depthFrame.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(convertedDepthBits, 0, ptr, 4 * depthFrame.PixelDataLength);
bmap.UnlockBits(bmapdata);
pictureBox2.Image = bmap;
}
}
}
private byte[] ConvertDepthFrame(short[] depthFrame, DepthImageStream depthStream)
{
//Run through the depth frame making the correlation between the two arrays
for (int i16 = 0, i32 = 0; i16 < depthFrame.Length && i32 < this.depthFrame32.Length; i16++, i32 += 4)
{
// Console.WriteLine(i16 + "," + i32);
//We don’t care about player’s information here, so we are just going to rule it out by shifting the value.
int realDepth = depthFrame[i16] >> DepthImageFrame.PlayerIndexBitmaskWidth;
//We are left with 13 bits of depth information that we need to convert into an 8 bit number for each pixel.
//There are hundreds of ways to do this. This is just the simplest one.
//Lets create a byte variable called Distance.
//We will assign this variable a number that will come from the conversion of those 13 bits.
byte Distance = 0;
//XBox Kinects (default) are limited between 800mm and 4096mm.
int MinimumDistance = 800;
int MaximumDistance = 4096;
//XBox Kinects (default) are not reliable closer to 800mm, so let’s take those useless measurements out.
//If the distance on this pixel is bigger than 800mm, we will paint it in its equivalent gray
if (realDepth > MinimumDistance)
{
//Convert the realDepth into the 0 to 255 range for our actual distance.
//Use only one of the following Distance assignments
//White = Far
//Black = Close
//Distance = (byte)(((realDepth – MinimumDistance) * 255 / (MaximumDistance-MinimumDistance)));
//White = Close
//Black = Far
Distance = (byte)(255 - ((realDepth - MinimumDistance) * 255 / (MaximumDistance - MinimumDistance)));
//Use the distance to paint each layer (R G & of the current pixel.
//Painting R, G and B with the same color will make it go from black to gray
this.depthFrame32[i32 + RedIndex] = (byte)(Distance);
this.depthFrame32[i32 + GreenIndex] = (byte)(Distance);
this.depthFrame32[i32 + BlueIndex] = (byte)(Distance);
}
//If we are closer than 800mm, the just paint it red so we know this pixel is not giving a good value
else
{
this.depthFrame32[i32 + RedIndex] = 0;
this.depthFrame32[i32 + GreenIndex] = 0;
this.depthFrame32[i32 + BlueIndex] = 0;
}
}
so i presume that rgb frame is working out for you in that case:
first to enable depth camera you need to call:
sensor->NuiInitialize(NUI_INITIALIZE_FLAG_USES_DEPTH|all stuff you use also);
second to start streaming you need to call:
if (int(streams&_Kinect_zed)) ret=sensor->NuiImageStreamOpen(
NUI_IMAGE_TYPE_DEPTH, // Depth camera or rgb camera?
NUI_IMAGE_RESOLUTION_640x480, // Image resolution
NUI_IMAGE_STREAM_FLAG_DISTINCT_OVERFLOW_DEPTH_VALUES, // Image stream flags // NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE nefunguje !!!
2, // Number of frames to buffer
NULL, // Event handle
&stream_hzed); else stream_hzed=NULL;
beware not all resolution/flags combinations work on all models of kinect !!!
this one above is safe even for the older models like mine
this is how i capture frame (called repeatedly from timer or thread loop)
ret=sensor->NuiImageStreamGetNextFrame(stream_hzed,0,&imageFrame); if (ret>=0)
{
// copy data from frame
imageFrame.pFrameTexture->LockRect(0, &LockedRect, NULL, 0);
if (LockedRect.Pitch!=0)
{
const BYTE* curr = (const BYTE*) LockedRect.pBits;
union _col { BYTE u8[2]; WORD u16; } col;
col.u16=0;
pnt3d p;
long ax,ay;
float mxs=float(xs)/(62.0*deg),mys=float(ys)/(48.6*deg);
for(int x=0,y=0;;)
{
col.u8[0]=*curr; curr++;
col.u8[1]=*curr; curr++;
p.raw=col.u16;
p.rgb=&rgb_default;
if (p.raw==0x0000) p.z=0.0; // p.z je kolma vzdialenost od senzora (kinect to correctuje sam)
else if (p.raw>=0x8000) p.z=4.0;
else p.z=0.8+(float(p.raw-6576)*0.00012115165336374002280501710376283);
// depth FOV correction
p.x=zx[x]*p.z;
p.y=zy[y]*p.z;
// color FOV correction zed 58.5° x 45.6° | rgb 62.0° x 48.6° | 25mm distance
if (p.z>0.0)
{
ax=(((x+10-xs2)*241)>>8)+xs2; // cameras x-offset and different FOV
ay=(((y+30-ys2)*240)>>8)+ys2; // cameras y-offset??? and different FOV
if ((ax>=0)&&(ax<xs))
if ((ay>=0)&&(ay<ys)) p.rgb=&rgb[ay][ax];
}
xyz[y][x]=p;
x++; if (x>=xs) { x=0; y++; if (y>=ys) break; }
}
}
// release frame
imageFrame.pFrameTexture->UnlockRect(0);
ret=sensor->NuiImageStreamReleaseFrame(stream_hzed, &imageFrame);
stream_changed|=_Kinect_zed;
}
Sorry for incomplete source code ...
- all is copy pasted from my kinect class (BDS2006 Turbo C++)
- so you need to check your code if you do not forget something
- and if yes then transform my code to C# (i am not C# user)
- most likely you forget to NUIinitialize with depth flag
- or set invalid resolution/flags/ precision or framerate for your HW
if nothing work at all then you need to initialize the sensor in the first place
int sensors;
INuiSensor *sensor;
if ((NUIGetSensorCount(&sensors)<0)||(sensors<1)) return false;
if (NUICreateSensorByIndex(0,&sensor)<0) return false;
if you link to dll on your own then link only these functions:
typedef HRESULT(__stdcall *_NuiGetSensorCount )(int * pCount); _NuiGetSensorCount NUIGetSensorCount =NULL;
typedef HRESULT(__stdcall *_NuiCreateSensorByIndex)(int index,INuiSensor **ppNuiSensor); _NuiCreateSensorByIndex NUICreateSensorByIndex=NULL;
Every other function (must) is obtained via COM inside SDK headers !!!
if you link and use them on your own then you will not be connected to your physical Kinect !!!
Basically kinect sdk is developed for WPf application. In windows form you have convert the short array of the depth data to the BItmap to display it on picturebox. And based on my expriment WPF is better for programming with kinect.
Below is the function that I used to convert depth frame to Bitmap for showing in picture box.
private Bitmap ImageToBitmap(DepthImageFrame Image)
{
short[] pixeldata = new short[Image.PixelDataLength];
int stride = Image.Width * 2;
Image.CopyPixelDataTo(pixeldata);
Bitmap bmap = new Bitmap(Image.Width, Image.Height, PixelFormat.Format16bppRgb555);
BitmapData bmapdata = bmap.LockBits(new Rectangle(0, 0, Image.Width, Image.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(pixeldata, 0, ptr, Image.PixelDataLength);
bmap.UnlockBits(bmapdata);
return bmap;
}
You may call it like this:
DepthImageFrame VFrame = e.OpenDepthImageFrame();
if (VFrame == null) return;
short[] pixelS = new short[VFrame.PixelDataLength];
Bitmap bmap = ImageToBitmap(VFrame);
I have a byte array that needs to be displayed on the desktop (or Form). I'm using WinApi for that and not sure how to set all pixels at once. The byte array is in my memory and needs to be displayed as quickly as possible (with just WinApi).
I'm using C# but simple pseudo-code would be ok for me:
// create bitmap
byte[] bytes = ...;// contains pixel data, 1 byte per pixel
HDC desktopDC = GetWindowDC(GetDesktopWindow());
HDC bitmapDC = CreateCompatibleDC(desktopDC);
HBITMAP bitmap = CreateCompatibleBitmap(bitmapDC, 320, 240);
DeleteObject(SelectObject(bitmapDC, bitmap));
BITMAPINFO info = new BITMAPINFO();
info.bmiColors = new tagRGBQUAD[256];
for (int i = 0; i < info.bmiColors.Length; i++)
{
info.bmiColors[i].rgbRed = (byte)i;
info.bmiColors[i].rgbGreen = (byte)i;
info.bmiColors[i].rgbBlue = (byte)i;
info.bmiColors[i].rgbReserved = 0;
}
info.bmiHeader = new BITMAPINFOHEADER();
info.bmiHeader.biSize = (uint) Marshal.SizeOf(info.bmiHeader);
info.bmiHeader.biWidth = 320;
info.bmiHeader.biHeight = 240;
info.bmiHeader.biPlanes = 1;
info.bmiHeader.biBitCount = 8;
info.bmiHeader.biCompression = BI_RGB;
info.bmiHeader.biSizeImage = 0;
info.bmiHeader.biClrUsed = 256;
info.bmiHeader.biClrImportant = 0;
// next line throws wrong parameter exception all the time
// SetDIBits(bitmapDC, bh, 0, 240, Marshal.UnsafeAddrOfPinnedArrayElement(info.bmiColors, 0), ref info, DIB_PAL_COLORS);
// how do i store all pixels into the bitmap at once ?
for (int i = 0; i < bytes.Length;i++)
SetPixel(bitmapDC, i % 320, i / 320, random(0x1000000));
// draw the bitmap
BitBlt(desktopDC, 0, 0, 320, 240, bitmapDC, 0, 0, SRCCOPY);
When I just try to set each pixel by itself with SetPixel() I see a monochrome image without gray colors only black and white. How can I correctly create a gray scale bitmap for displaying ? And how do I do that quick ?
Update:
The call ends up in an error outside of my program in WinApi. Can't catch exception:
public const int DIB_RGB_COLORS = 0;
public const int DIB_PAL_COLORS = 1;
[DllImport("gdi32.dll")]
public static extern int SetDIBits(IntPtr hdc, IntPtr hbmp, uint uStartScan, uint cScanLines, byte[] lpvBits, [In] ref BITMAPINFO lpbmi, uint fuColorUse);
// parameters as above
SetDIBits(bitmapDC, bitmap, 0, 240, bytes, ref info, DIB_RGB_COLORS);
Two of the SetDIBits parameters are wrong:
lpvBits - this is the image data but you're passing the palette data. You should be passing your bytes array.
lpBmi - this is OK - the BITMAPINFO structure contains both the BITMAPINFOHEADER and the palette so you don't need to pass the palette separately. My answer to your other question describes how to declare the structure.
fuColorUse - this describes the format of the palette. You are using an RGB palette so you should pass DIB_RGB_COLORS.
I am trying to increase my image detection class using lockbits, yet this cause problems with the code and thus it does not run. How can i go about using lockbits and getpixel at the same time in order to speed up image detection, or can someone show me an alternative which is just as fast?
code:
static IntPtr Iptr = IntPtr.Zero;
static BitmapData bitmapData = null;
static public byte[] Pixels { get; set; }
static public int Depth { get; private set; }
static public int Width { get; private set; }
static public int Height { get; private set; }
static public void LockBits(Bitmap source)
{
// Get width and height of bitmap
Width = source.Width;
Height = source.Height;
// get total locked pixels count
int PixelCount = Width * Height;
// Create rectangle to lock
Rectangle rect = new Rectangle(0, 0, Width, Height);
// get source bitmap pixel format size
Depth = System.Drawing.Bitmap.GetPixelFormatSize(source.PixelFormat);
// Lock bitmap and return bitmap data
bitmapData = source.LockBits(rect, ImageLockMode.ReadWrite,
source.PixelFormat);
// create byte array to copy pixel values
int step = Depth / 8;
Pixels = new byte[PixelCount * step];
Iptr = bitmapData.Scan0;
// Copy data from pointer to array
Marshal.Copy(Iptr, Pixels, 0, Pixels.Length);
}
static public bool SimilarColors(int R1, int G1, int B1, int R2, int G2, int B2, int Tolerance)
{
bool returnValue = true;
if (Math.Abs(R1 - R2) > Tolerance || Math.Abs(G1 - G2) > Tolerance || Math.Abs(B1 - B2) > Tolerance)
{
returnValue = false;
}
return returnValue;
}
public bool findImage(Bitmap small, Bitmap large, out Point location)
{
unsafe
{
LockBits(small);
LockBits(large);
//Loop through large images width
for (int largeX = 0; largeX < large.Width; largeX++)
{
//And height
for (int largeY = 0; largeY < large.Height; largeY++)
{
//Loop through the small width
for (int smallX = 0; smallX < small.Width; smallX++)
{
//And height
for (int smallY = 0; smallY < small.Height; smallY++)
{
//Get current pixels for both image
Color currentSmall = small.GetPixel(smallX, smallY);
Color currentLarge = large.GetPixel(largeX + smallX, largeY + smallY);
//If they dont match (i.e. the image is not there)
if (!colorsMatch(currentSmall, currentLarge))
//Goto the next pixel in the large image
goto nextLoop;
}
}
//If all the pixels match up, then return true and change Point location to the top left co-ordinates where it was found
location = new Point(largeX, largeY);
return true;
//Go to next pixel on large image
nextLoop:
continue;
}
}
//Return false if image is not found, and set an empty point
location = Point.Empty;
return false;
}
}
You wouldn't want to rely on getPixel() for image processing; it's okay to make an occasional call to get a point value (e.g. on mouseover), but in general it's preferable to do image processing in image memory or in some 2D array that you can convert to a Bitmap when necessary.
To start, you might try writing a method that using LockBits/UnlockBits to extract an array that is convenient to manipulate. Once you're done manipulating the array, you can write it back to a bitmap using a different LockBits/UnlockBits function.
Here's some sample code I've used in the past. The first function returns a 1D array of values from a Bitmap. Since you know the bitmap's width, you can convert this 1D array to a 2D array for further processing. Once you're done processing, you can call the second function to convert the (modified) 1D array into a bitmap again.
public static byte[] Array1DFromBitmap(Bitmap bmp){
if (bmp == null) throw new NullReferenceException("Bitmap is null");
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
BitmapData data = bmp.LockBits(rect, ImageLockMode.ReadWrite, bmp.PixelFormat);
IntPtr ptr = data.Scan0;
//declare an array to hold the bytes of the bitmap
int numBytes = data.Stride * bmp.Height;
byte[] bytes = new byte[numBytes];
//copy the RGB values into the array
System.Runtime.InteropServices.Marshal.Copy(ptr, bytes, 0, numBytes);
bmp.UnlockBits(data);
return bytes;
}
public static Bitmap BitmapFromArray1D(byte[] bytes, int width, int height)
{
Bitmap grayBmp = new Bitmap(width, height, PixelFormat.Format8bppIndexed);
Rectangle grayRect = new Rectangle(0, 0, grayBmp.Width, grayBmp.Height);
BitmapData grayData = grayBmp.LockBits(grayRect, ImageLockMode.ReadWrite, grayBmp.PixelFormat);
IntPtr grayPtr = grayData.Scan0;
int grayBytes = grayData.Stride * grayBmp.Height;
ColorPalette pal = grayBmp.Palette;
for (int g = 0; g < 256; g++){
pal.Entries[g] = Color.FromArgb(g, g, g);
}
grayBmp.Palette = pal;
System.Runtime.InteropServices.Marshal.Copy(bytes, 0, grayPtr, grayBytes);
grayBmp.UnlockBits(grayData);
return grayBmp;
}
These methods makes assumptions about the Bitmap pixel format that may not work for you, but I hope the general idea is clear: use LockBits/UnlockBits to extract an array of bytes from a Bitmap so that you can write and debug algorithms most easily, and then use LockBits/UnlockBits again to write the array to a Bitmap again.
For portability, I would recommend that your methods return the desired data types rather than manipulating global variables within the methods themselves.
If you've been using getPixel(), then converting to/from arrays as described above could speed up your code considerably for a small investment of coding effort.
Ok where to start. Better you understand what you are doing with lockBits.
First of all make sure, that you dont overwrite your byte array with.
LockBits(small);
LockBits(large);
due to the second call all the first call does is locking your image and that is not good since you doesn't unlock it again.
So add another byte array that represents the image.
You can do something like this
LockBits(small, true);
LockBits(large, false);
and change your Lockbits method
static public void LockBits(Bitmap source, bool flag)
{
...
Marshal.Copy(Iptr, Pixels, 0, Pixels.Length);
if(flag)
PixelsSmall=Pixels;
else
PixelsLarge=Pixels;
}
where PixelsLarge and PixelsSmall are globals and Pixels isn't
Those 2 contain your image. Now you have to compare it.
Now you have to compare each "set of bytes" therefore you have to know the Pixelformat.
Is it 32b/pix 24 or only 8 (ARGB,RGB,grayscale)
Let's take ARGB images. In this case a set would consist of 4 bytes (=32/8)
I am not sure about the order but I think the order of one set is ABGR or BGRA.
Hope this may help you. If you don't figure out how to compare the right pixels then ask again. Ah and dont forget to use the UnlockBits command.