Develop Reference

Getting the screen pixels as byte array

I need to change my screen capture code to get a pixel array instead of a Bitmap.
I change the code to this:
BitBlt > Image.FromHbitmap(pointer) > LockBits > pixel array
But, I'm checking if it's possible to cut some middle man, and have something like this:
BitBlt > Marshal.Copy > pixel array
Or even:
WinApi method that gets the screen region as a pixel array
So far, I tried to use this code, without success:
public static byte[] CaptureAsArray(Size size, int positionX, int positionY)
{
var hDesk = GetDesktopWindow();
var hSrce = GetWindowDC(hDesk);
var hDest = CreateCompatibleDC(hSrce);
var hBmp = CreateCompatibleBitmap(hSrce, (int)size.Width, (int)size.Height);
var hOldBmp = SelectObject(hDest, hBmp);
try
{
new System.Security.Permissions.UIPermission(System.Security.Permissions.UIPermissionWindow.AllWindows).Demand();
var b = BitBlt(hDest, 0, 0, (int)size.Width, (int)size.Height, hSrce, positionX, positionY, CopyPixelOperation.SourceCopy | CopyPixelOperation.CaptureBlt);
var length = 4 * (int)size.Width * (int)size.Height;
var bytes = new byte[length];
Marshal.Copy(hBmp, bytes, 0, length);
//return b ? Image.FromHbitmap(hBmp) : null;
return bytes;
}
finally
{
SelectObject(hDest, hOldBmp);
DeleteObject(hBmp);
DeleteDC(hDest);
ReleaseDC(hDesk, hSrce);
}
return null;
}
This code gives me an System.AccessViolationException while stepping on Marshal.Copy.
Is there any more efficient way of getting screen pixels as a byte array while using BitBlt or similar screen capture methods?
EDIT:
As found in here and as suggested by CodyGray, I should use
var b = Native.BitBlt(_compatibleDeviceContext, 0, 0, Width, Height, _windowDeviceContext, Left, Top, Native.CopyPixelOperation.SourceCopy | Native.CopyPixelOperation.CaptureBlt);
var bi = new Native.BITMAPINFOHEADER();
bi.biSize = (uint)Marshal.SizeOf(bi);
bi.biBitCount = 32;
bi.biClrUsed = 0;
bi.biClrImportant = 0;
bi.biCompression = 0;
bi.biHeight = Height;
bi.biWidth = Width;
bi.biPlanes = 1;
var data = new byte[4 * Width * Height];
Native.GetDIBits(_windowDeviceContext, _compatibleBitmap, 0, (uint)Height, data, ref bi, Native.DIB_Color_Mode.DIB_RGB_COLORS);
My data array has all the pixels of the screenshot.
Now, I'm going to test if there's any performance improvements or not.

Yeah, you can't just start accessing the raw bits of a BITMAP object through an HBITMAP (as returned by CreateCompatibleBitmap). HBITMAP is just a handle, as the name suggests. It's not a pointer in the classic "C" sense that it points to the beginning of the array. Handles are like indirect pointers.
GetDIBits is the appropriate solution to get the raw, device-independent pixel array from a bitmap that you can iterate through. But you'll still need to use the code you have to get the screen bitmap in the first place. Essentially, you want something like this. Of course, you'll need to translate it into C#, but that shouldn't be difficult, since you already know how to call WinAPI functions.
Note that you do not need to call GetDesktopWindow or GetWindowDC. Just pass NULL as the handle to GetDC; it has the same effect of returning a screen DC, which you can then use to create a compatible bitmap. In general, you should almost never call GetDesktopWindow.

OpenTK Getting pixels byte array from a Bitmap

I need to port some OpenGL code to C# OpenTK.
Here is the chunk where I update a mapped PBO from an array of pixels in C++ :
GLubyte* ptr = (GLubyte*)glMapBuffer(GL_PIXEL_UNPACK_BUFFER, GL_WRITE_ONLY);
if(ptr)
{
memcpy(ptr,imageInfo.Data,IMG_DATA_SIZE);
glUnmapBuffer(GL_PIXEL_UNPACK_BUFFER);
}
I need to do the same in OpenTK.My image data comes from an instance of Bitmap.
I tried the following:
IntPtr ptr = GL.MapBuffer(BufferTarget.PixelUnpackBuffer, BufferAccess.WriteOnly);
if(ptr != IntPtr.Zero)
{
BitmapData data = updateColorMap.LockBits(new System.Drawing.Rectangle(0, 0, updateColorMap.Width, updateColorMap.Height),
ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
Marshal.Copy(data.Scan0, 0, ptr, IMG_DATA_SIZE);
}
But Marshal.Copy requires the first param to be of byte[] type.I didn't find how to retrieve it from the BitmapData.It returns only IntPtr (data.Scan0) .
So how can I get the byte array from the Bitmap?
UPDATE:
In the meantime I got help from the OpenTK forum and they proposed to do this instead:
unsafe
{
GL.BufferData(BufferTarget.PixelUnpackBuffer, new IntPtr(IMG_DATA_SIZE), IntPtr.Zero, BufferUsageHint.StreamDraw);
byte* ptr = (byte*)GL.MapBuffer(BufferTarget.PixelUnpackBuffer, BufferAccess.WriteOnly);
if (ptr != null)
{
BitmapData data = updateDepthMap.LockBits(new System.Drawing.Rectangle(0, 0, updateDepthMap.Width, updateDepthMap.Height),
ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
byte* scan0 = (byte*)data.Scan0.ToPointer();
for (int i = 0; i < IMG_DATA_SIZE; ++i)
{
*ptr = *scan0;
++ptr;
++scan0;
}
updateDepthMap.UnlockBits(data);
GL.UnmapBuffer(BufferTarget.PixelUnpackBuffer);
}
}//unsafe
Now,this works,but it is TERRIBLY SLOW! The regular texture update runs 2x faster than this,which is
wrong as async PBO transfer should speed up texture uploads.Indeed in my C++ version PBO upload causes almost 2x performance boost.

Ok so the solution is here: Copy data from from IntPtr to IntPtr
tested on linux.

Memory Leak when Marshal.Copy

I noticed my program was leaking memory. So I used dotMemory to find the leak, and looks like this is the function causing the leak:
private void LoadBits()
{
// Lock the bitmap's bits.
Rectangle rect = new Rectangle(0, 0, bm.Width, bm.Height);
bmpData = bm.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite, bm.PixelFormat);
stride = bmpData.Stride;
// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;
// Declare an array to hold the bytes of the bitmap.
byteCount = Math.Abs(bmpData.Stride) * bm.Height;
bytes = new byte[byteCount];
// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, bytes, 0, byteCount);
}
And this how I unlock the bits.
private void SaveBits()
{
// Update Stuff
IntPtr ptr = bmpData.Scan0;
System.Runtime.InteropServices.Marshal.Copy(bytes, 0, ptr, byteCount);
bm.UnlockBits(bmpData);
}
I implemented the IDisposable interface for this class. And I call the SaveBits there, so even if I forget to call SaveBits, the GC should do it for me.
And Yes, I do call bm.Dispose() and set everything to null in the Dispose method.

You need to UnlockBits() when you're done.

System::Drawing::Bitmap to unsigned char *

I have a C# .NET library that grabs frames from a camera. I need to send those frames to a native application that takes images from an unsigned char*.
I initially take the frames as System::Drawing::Bitmap.
So far I can retrieve a byte[] from the Bitmap. My test is done with an image of resolution 400*234, I should be getting 400*234*3 bytes to get to the 24bpp a RGB image requires.
However, I'm getting a byte[] of size 11948.
This is how I convert from Bitmap to byte[]:
private static byte[] ImageToByte(Bitmap img)
{
ImageConverter converter = new ImageConverter();
return (byte[])converter.ConvertTo(img, typeof(byte[]));
}
What is the proper way to convert from System::Drawing::Bitmap to RGB unsigned char*?

This has to be done using the lockBits method, here is a code example:
Rectangle rect = new Rectangle(0, 0, m_bitmap.Width, m_bitmap.Height);
BitmapData bmpData = m_bitmap.LockBits(rect, ImageLockMode.ReadOnly,
m_bitmap.PixelFormat);
IntPtr ptr = bmpData.Scan0;
int bytes = Math.Abs(bmpData.Stride) * m_bitmap.Height;
byte[] rgbValues = new byte[bytes];
Marshal.Copy(ptr, rgbValues, 0, bytes);
m_bitmap.UnlockBits(bmpData);
GCHandle handle = GCHandle::Alloc(rgbValues, GCHandleType::Pinned);
unsigned char * data = (unsigned char*) (void*) handle.AddrOfPinnedObject();
//do whatever with data

Copy a channel from one image to another using safe code

I'm trying to refactor this unsafe code to copy a single ARGB channel from one image to another using System.Runtime.InteropServices.Marshal.Copy as per this example on MSDN but I'm totally lost.
Could anyone walk me through how I would go about it?
public enum ChannelARGB
{
Blue = 0,
Green = 1,
Red = 2,
Alpha = 3
}
public static void transferOneARGBChannelFromOneBitmapToAnother(
Bitmap source,
Bitmap dest,
ChannelARGB sourceChannel,
ChannelARGB destChannel )
{
if ( source.Size!=dest.Size )
throw new ArgumentException();
Rectangle r = new Rectangle( Point.Empty, source.Size );
BitmapData bdSrc = source.LockBits( r,
ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb );
BitmapData bdDst = dest.LockBits( r,
ImageLockMode.ReadWrite,
PixelFormat.Format32bppArgb );
unsafe
{
byte* bpSrc = (byte*)bdSrc.Scan0.ToPointer();
byte* bpDst = (byte*)bdDst.Scan0.ToPointer();
bpSrc += (int)sourceChannel;
bpDst += (int)destChannel;
for ( int i = r.Height * r.Width; i > 0; i-- )
{
*bpDst = *bpSrc;
bpSrc += 4;
bpDst += 4;
}
}
source.UnlockBits( bdSrc );
dest.UnlockBits( bdDst );
}
Edit
In an attempt to work through #Ben Voigt walk though I have come up with this so far. Unfortunately I am now getting the following error:
Attempted to read or write protected memory. This is often an
indication that other memory is corrupt.
private static void TransferOneArgbChannelFromOneBitmapToAnother(
Bitmap source,
Bitmap destination,
ChannelARGB sourceChannel,
ChannelARGB destinationChannel)
{
if (source.Size != destination.Size)
{
throw new ArgumentException();
}
Rectangle rectangle = new Rectangle(Point.Empty, source.Size);
// Lockbits the source.
BitmapData bitmapDataSource = source.LockBits(rectangle,
ImageLockMode.ReadWrite,
PixelFormat.Format32bppArgb);
// Declare an array to hold the bytes of the bitmap.
int bytes = bitmapDataSource.Stride * bitmapDataSource.Height;
// Allocate a buffer for the source image
byte[] sourceRgbValues = new byte[bytes];
// Get the address of the first line.
IntPtr ptrSource = bitmapDataSource.Scan0;
// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptrSource,
sourceRgbValues,
0,
bytes);
// Unlockbits the source.
source.UnlockBits(bitmapDataSource);
// Lockbits the destination.
BitmapData bitmapDataDestination = destination.LockBits(rectangle,
ImageLockMode.ReadWrite,
PixelFormat.Format32bppArgb);
// Allocate a buffer for image
byte[] destinationRgbValues = new byte[bytes];
IntPtr ptrDestination = bitmapDataDestination.Scan0;
// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptrDestination,
destinationRgbValues,
0,
bytes);
ptrSource += (int)sourceChannel;
ptrDestination += (int)destinationChannel;
for (int i = rectangle.Height * rectangle.Width; i > 0; i--)
{
destinationRgbValues[i] = sourceRgbValues[i];
ptrSource += 4;
ptrDestination += 4;
}
// Copy the RGB values back to the bitmap
// ******This is where I am getting the exception*******.
System.Runtime.InteropServices.Marshal.Copy(destinationRgbValues,
0,
ptrDestination,
bytes);
// Unlock bits the destination.
destination.UnlockBits(bitmapDataDestination);
}
Can anyone see what I have done wrong? This is all a bit over my head to be honest. I think I should buy some books.

LockBits the source.
Marshal.Copy the source BitmapData to a byte[] buffer.
UnlockBits the source.
LockBits the destination.
Marshal.Copy the destination BitmapData to a byte[] buffer.
Loop through and copy that channel from the source byte[] to the destination byte[] (note, use arithmetic on indexes instead of on pointers)
Marshal.Copy the destination byte[] back to the BitmapData.
UnlockBits the destination.
I'm not sure what the point is, though. Code that uses Marshal.Copy is just as dangerous as code that uses the unsafe keyword, and should require similar code security permission.
A potentially more efficient way would be to use ImageAttributes.SetColorMatrix to remove the desired channel from the destination image, remove all other channels from the source image, and then blend. See the example for ColorMatrix
Or use DirectX (or OpenGL) and a shader that just transfers the one channel.

You could use my simple LINQ based image processing framework from Nuget or Codeplex and write a simple query that swaps the channels around.
You could also use a ColorMatrix to perform the channel swap like in this code.

Unfortunately, a ColorMatrix won't work if you want to combine channels from two separate images. You would need an additive (or bitwise or) blending method, and the only blending provided by GDI+ is Over and Copy. It also looks to me like any methods that would allow you to access the bits directly, including LockBits, are locked down.
I think the only option is to use GetPixel and SetPixel on each pixel, something like this:
Color dstColor = bpDst.GetPixel(x, y);
Color srcColor = bpSrc.GetPixel(x, y);
int srcValue = (srcColor.ToArgb() >> (sourceChannel * 8)) & 0xff;
int dstArgb = (dstColor.ToArgb() & ~(0xff << (destChannel * 8))) | (srcValue << (destChannel * 8));
bpDst.SetPixel(x, y, Color.FromArgb(dstArgb));