I'm trying to get cursor data (mask and color, if available) as a byte array using GetDIBits(), with the correct size and correct animation frame but it's proving to be unreliable.
Also, I would like to get or be able to simulate the multiple frames available in a cursor, such as what is possible when using DrawIconEx(), passing the cursorStep parameter.
Basically what I'm trying to emulate is part of the behavior of IDXGIOutputDuplication::GetFramePointerShape(), to get a buffer (byte[]) with the contents of the current cursor, just without having to rely on the DirectX library.
So, I want:
Get the cursor data as a byte[], even with the mask working (SOLVED).
Get the cursor data with the correct size, not a fixed 32x32px cursor.
Get the actual animation frame of the cursor.
Just as the DXGI method provides if I'm using OutputDuplication, but in my case, I'm not.
1) Get the cursor mask correctly (solved):
TL;DR: Wrong data interpretation on my part.
I'm calling GetDIBits() once, passing null as the buffer parameter to get the details of the color/mask images and a second time to get the image.
It works all right with the color image but returns a wrong data size and wrong image content for the mask image.
Here's a comparison and the code sample.
Files ending with "2" were created by using System.Drawing.Bitmap.FromHBitmap() while the others were obtained from GetDiBits().
For some reason, GetDIBits() returns this data (BitmapInfoHeader) for the mask. Also, only returns 128 bytes (256 if the mask has two frames, such as for the I-Beam/Text cursor), which is way too little for 32x32px or 32x64px masks.
Here's my code so far (it's C#):
WindowDeviceContext = User32.GetWindowDC(IntPtr.Zero);
//The parameter passed in the structs is just to be able to calculate the size of it.
var cursorInfo = new CursorInfo(false);
if (!User32.GetCursorInfo(out cursorInfo))
return;
if (cursorInfo.Flags != Native.Constants.CursorShowing)
{
Gdi32.DeleteObject(cursorInfo.CursorHandle);
return;
}
var iconHandle = User32.CopyIcon(cursorInfo.CursorHandle);
if (iconHandle == IntPtr.Zero)
{
User32.DestroyIcon(iconHandle);
Gdi32.DeleteObject(cursorInfo.CursorHandle);
return;
}
if (!User32.GetIconInfo(iconHandle, out var iconInfo))
{
Gdi32.DeleteObject(iconInfo.Color);
Gdi32.DeleteObject(iconInfo.Mask);
User32.DestroyIcon(iconHandle);
Gdi32.DeleteObject(cursorInfo.CursorHandle);
return;
}
//Color.
var colorHeader = new BitmapInfoHeader(false);
//Gets the color image details.
Gdi32.GetDIBits(WindowDeviceContext, iconInfo.Color, 0, 0, null, ref colorHeader, DibColorModes.RgbColors);
if (colorHeader.Height != 0)
{
colorHeader.Height *= -1;
var colorBuffer = new byte[colorHeader.SizeImage];
Gdi32.GetDIBits(WindowDeviceContext, iconInfo.Color, 0, (uint)(colorHeader.Height * -1), colorBuffer, ref colorHeader, DibColorModes.RgbColors);
//Converts a byte array to image (System.Windows.Media.Imaging.BitmapSource) just for testing.
var image = ImageUtil.ImageMethods.FromArray(colorBuffer, colorHeader.Width, colorHeader.Height * -1, 4); //4 channels
using var fileStream = new FileStream(#"Color.png", FileMode.Create);
var bmpEncoder = new PngBitmapEncoder();
bmpEncoder.Frames.Add(BitmapFrame.Create(image));
bmpEncoder.Save(fileStream);
//Converts the HBitmap to Bitmap, just for testing too.
var image2 = System.Drawing.Image.FromHbitmap(iconInfo.Color);
image2.Save(#"Color2.png");
}
//Mask.
var maskHeader = new BitmapInfoHeader(false);
//Gets the mask image details.
Gdi32.GetDIBits(WindowDeviceContext, iconInfo.Mask, 0, 0, null, ref maskHeader, DibColorModes.RgbColors);
if (maskHeader.Height != 0)
{
maskHeader.Height *= -1;
var maskBuffer = new byte[maskHeader.SizeImage];
Gdi32.GetDIBits(WindowDeviceContext, iconInfo.Mask, 0, (uint) maskHeader.Height, maskBuffer, ref maskHeader, DibColorModes.RgbColors);
//Converts a byte array to image (System.Windows.Media.Imaging.BitmapSource) just for testing.
var image = ImageUtil.ImageMethods.FromArray(maskBuffer.ToList(), maskHeader.Width, maskHeader.Height * -1, 1, 1);
using var fileStream = new FileStream(#"Mask.png", FileMode.Create);
var bmpEncoder = new PngBitmapEncoder();
bmpEncoder.Frames.Add(BitmapFrame.Create(image));
bmpEncoder.Save(fileStream);
//Converts the HBitmap to Bitmap, just for testing too.
var image2 = System.Drawing.Image.FromHbitmap(iconInfo.Mask);
image2.Save(#"Mask2.png");
}
Gdi32.DeleteObject(iconInfo.Color);
Gdi32.DeleteObject(iconInfo.Mask);
User32.DestroyIcon(iconHandle);
Gdi32.DeleteObject(cursorInfo.CursorHandle);
Solved:
With the proper interpretation of the bits per pixel of the mask image, this issue was solved.
public static BitmapSource FromArray(byte[] data, int width, int height, int channels, int bitsPerPixel = 32)
{
var format = PixelFormats.Default;
var stride = channels * ((bitsPerPixel * width + 31) / 32);
//Abridged channel and bits by pixel to format converter.
if (channels == 1)
{
format = PixelFormats.BlackWhite;
stride = width / 8;
}
else if (channels == 3)
format = PixelFormats.Bgr24; //RGB.
else if (channels == 4)
format = PixelFormats.Bgr32; //RGB + alpha.
var wbm = new WriteableBitmap(width, height, 96, 96, format, null);
wbm.WritePixels(new Int32Rect(0, 0, width, height), data, stride, 0);
return wbm;
}
2) Get the correct size of the cursor:
As noted above, I'm only getting cursor images of size 32x32 or 32x64 (for masks).
I've tried to increase the size being passed to GetDIBits(), but the image being returned is always the same size.
Even if the size in Windows 10/11 settings is set to a bigger cursor size.
The IDXGIOutputDuplication::GetFramePointerShape() method returns a correct size, so it looks like it's possible. What does this method do in order to get the correct image size?
3) Get the actual cursor frame being displayed (for animated cursors):
As for the animation frame (steps) of the cursor, I don't see a way of getting multiple frames based on an index the same way that I would be able to draw with DrawIconEx():
//If the cursor rate needs to be precisely captured, I could use this undocumented API:
//https://source.winehq.org/source/dlls/user32/cursoricon.c#2325
//int rate = 0, num = 0;
//var ok1 = User32.GetCursorFrameInfo(cursorInfo.hCursor, IntPtr.Zero, 17, ref rate, ref num);
//CursorStep
var ok = User32.DrawIconEx(CompatibleDeviceContext, frame.CursorX - iconInfo.XHotspot, frame.CursorY - iconInfo.YHotspot, cursorInfo.CursorHandle, 0, 0, CursorStep, IntPtr.Zero, 0x0003);
if (!ok)
{
CursorStep = 0;
User32.DrawIconEx(CompatibleDeviceContext, frame.CursorX - iconInfo.XHotspot, frame.CursorY - iconInfo.YHotspot, cursorInfo.CursorHandle, 0, 0, CursorStep, IntPtr.Zero, 0x0003);
}
else
CursorStep++;
Maybe I could draw the cursor color and masks separately using DrawIconEx() into two bitmaps (using the flags DI_IMAGE and DI_MASK in each call, passing the cursor step), and then getting the pixels using GetDIBits()?
But how to do that?
Related
I'm developing an application to concatenate a bitmap image in RGB with a TIFF in CMYK.
I've tried with System.Drawing and System.Windows.Media namespaces.
The problem is both the libraries try to convert my TIFF image into RGB before merging, which causes a loss in image quality.
As far as I understand, the reason they always convert images into RGB before processing because the two libraries do that with a rendering intent.
I don't need to render anything, just merge the two photos and save to disk, that's all.
What should I do to achieve my goal? Clearly, I don't want to lose the quality of the TIFF so I think it's best to not do any conversion, just keep it raw and merge. Anyway, that's just a guess, other option could be considered as well. Could anybody shed some light on my case please?
See a comparison of the tiff image before and after converted from cmyk to rgb below.
I’m not aware of any capacity in the TIFF format to have two different color spaces at the same time. Since you are dealing in CMYK, I assume that is the one you want to preserve.
If so, the steps to do so would be:
Load CMYK image A (using BitmapDecoder)
Load RGB image B (using BitmapDecoder)
Convert image B to CMYK with the desired color profile (using FormatConvertedBitmap)
If required, ensure the pixel format for image B matches A (using FormatConvertedBitmap)
Composite the two in memory as a byte array (using CopyPixels, then memory manipulation, then new bitmap from the memory)
Save the composite to a new CMYK TIFF file (using TiffBitmapEncoder)
That should be possible with WIC (System.Media).
An example doing so (github) could be written as:
BitmapFrame LoadTiff(string filename)
{
using (var rs = File.OpenRead(filename))
{
return BitmapDecoder.Create(rs, BitmapCreateOptions.PreservePixelFormat, BitmapCacheOption.OnLoad).Frames[0];
}
}
// Load, validate A
var imageA = LoadTiff("CMYK.tif");
if (imageA.Format != PixelFormats.Cmyk32)
{
throw new InvalidOperationException("imageA is not CMYK");
}
// Load, validate, convert B
var imageB = LoadTiff("RGB.tif");
if (imageB.PixelHeight != imageA.PixelHeight)
{
throw new InvalidOperationException("Image B is not the same height as image A");
}
var imageBCmyk = new FormatConvertedBitmap(imageB, imageA.Format, null, 0d);
// Merge
int width = imageA.PixelWidth + imageB.PixelWidth,
height = imageA.PixelHeight,
bytesPerPixel = imageA.Format.BitsPerPixel / 8,
stride = width * bytesPerPixel;
var buffer = new byte[stride * height];
imageA.CopyPixels(buffer, stride, 0);
imageBCmyk.CopyPixels(buffer, stride, imageA.PixelWidth * bytesPerPixel);
var result = BitmapSource.Create(width, height, imageA.DpiX, imageA.DpiY, imageA.Format, null, buffer, stride);
// save to new file
using (var ws = File.Create("out.tif"))
{
var tiffEncoder = new TiffBitmapEncoder();
tiffEncoder.Frames.Add(BitmapFrame.Create(result));
tiffEncoder.Save(ws);
}
Which maintains color accuracy of the CMYK image, and converts the RGB using the system color profile. This can be verified in Photoshop which shows that the each letter, and rich black, have maintained their original values. (note that imgur does convert to png with dubious color handling - check github for originals.)
Image A (CMYK):
Image B (RGB):
Result (CMYK):
To have the two images overlayed, one image would have to have some notion of transparency. A mask would be one example thereof, where you pick a particular color value to mean "transparent". The downside of a mask is that masks do not play well with aliased source images. For that, you would want to do an alpha channel - but blending across color spaces would be challenging. (Github)
// Load, validate A
var imageA = LoadTiff("CMYK.tif");
if (imageA.Format != PixelFormats.Cmyk32)
{
throw new InvalidOperationException("imageA is not CMYK");
}
// Load, validate, convert B
var imageB = LoadTiff("RGBOverlay.tif");
if (imageB.PixelHeight != imageA.PixelHeight
|| imageB.PixelWidth != imageA.PixelWidth)
{
throw new InvalidOperationException("Image B is not the same size as image A");
}
var imageBBGRA = new FormatConvertedBitmap(imageB, PixelFormats.Bgra32, null, 0d);
var imageBCmyk = new FormatConvertedBitmap(imageB, imageA.Format, null, 0d);
// Merge
int width = imageA.PixelWidth, height = imageA.PixelHeight;
var stride = width * (imageA.Format.BitsPerPixel / 8);
var bufferA = new uint[width * height];
var bufferB = new uint[width * height];
var maskBuffer = new uint[width * height];
imageA.CopyPixels(bufferA, stride, 0);
imageBBGRA.CopyPixels(maskBuffer, stride, 0);
imageBCmyk.CopyPixels(bufferB, stride, 0);
for (int i = 0; i < bufferA.Length; i++)
{
// set pixel in bufferA to the value from bufferB if mask is not white
if (maskBuffer[i] != 0xffffffff)
{
bufferA[i] = bufferB[i];
}
}
var result = BitmapSource.Create(width, height, imageA.DpiX, imageA.DpiY, imageA.Format, null, bufferA, stride);
// save to new file
using (var ws = File.Create("out_overlay.tif"))
{
var tiffEncoder = new TiffBitmapEncoder();
tiffEncoder.Frames.Add(BitmapFrame.Create(result));
tiffEncoder.Save(ws);
}
Example image B:
Example output:
I need to change my screen capture code to get a pixel array instead of a Bitmap.
I change the code to this:
BitBlt > Image.FromHbitmap(pointer) > LockBits > pixel array
But, I'm checking if it's possible to cut some middle man, and have something like this:
BitBlt > Marshal.Copy > pixel array
Or even:
WinApi method that gets the screen region as a pixel array
So far, I tried to use this code, without success:
public static byte[] CaptureAsArray(Size size, int positionX, int positionY)
{
var hDesk = GetDesktopWindow();
var hSrce = GetWindowDC(hDesk);
var hDest = CreateCompatibleDC(hSrce);
var hBmp = CreateCompatibleBitmap(hSrce, (int)size.Width, (int)size.Height);
var hOldBmp = SelectObject(hDest, hBmp);
try
{
new System.Security.Permissions.UIPermission(System.Security.Permissions.UIPermissionWindow.AllWindows).Demand();
var b = BitBlt(hDest, 0, 0, (int)size.Width, (int)size.Height, hSrce, positionX, positionY, CopyPixelOperation.SourceCopy | CopyPixelOperation.CaptureBlt);
var length = 4 * (int)size.Width * (int)size.Height;
var bytes = new byte[length];
Marshal.Copy(hBmp, bytes, 0, length);
//return b ? Image.FromHbitmap(hBmp) : null;
return bytes;
}
finally
{
SelectObject(hDest, hOldBmp);
DeleteObject(hBmp);
DeleteDC(hDest);
ReleaseDC(hDesk, hSrce);
}
return null;
}
This code gives me an System.AccessViolationException while stepping on Marshal.Copy.
Is there any more efficient way of getting screen pixels as a byte array while using BitBlt or similar screen capture methods?
EDIT:
As found in here and as suggested by CodyGray, I should use
var b = Native.BitBlt(_compatibleDeviceContext, 0, 0, Width, Height, _windowDeviceContext, Left, Top, Native.CopyPixelOperation.SourceCopy | Native.CopyPixelOperation.CaptureBlt);
var bi = new Native.BITMAPINFOHEADER();
bi.biSize = (uint)Marshal.SizeOf(bi);
bi.biBitCount = 32;
bi.biClrUsed = 0;
bi.biClrImportant = 0;
bi.biCompression = 0;
bi.biHeight = Height;
bi.biWidth = Width;
bi.biPlanes = 1;
var data = new byte[4 * Width * Height];
Native.GetDIBits(_windowDeviceContext, _compatibleBitmap, 0, (uint)Height, data, ref bi, Native.DIB_Color_Mode.DIB_RGB_COLORS);
My data array has all the pixels of the screenshot.
Now, I'm going to test if there's any performance improvements or not.
Yeah, you can't just start accessing the raw bits of a BITMAP object through an HBITMAP (as returned by CreateCompatibleBitmap). HBITMAP is just a handle, as the name suggests. It's not a pointer in the classic "C" sense that it points to the beginning of the array. Handles are like indirect pointers.
GetDIBits is the appropriate solution to get the raw, device-independent pixel array from a bitmap that you can iterate through. But you'll still need to use the code you have to get the screen bitmap in the first place. Essentially, you want something like this. Of course, you'll need to translate it into C#, but that shouldn't be difficult, since you already know how to call WinAPI functions.
Note that you do not need to call GetDesktopWindow or GetWindowDC. Just pass NULL as the handle to GetDC; it has the same effect of returning a screen DC, which you can then use to create a compatible bitmap. In general, you should almost never call GetDesktopWindow.
I have recently started using openCL.Net and have run into an issue with some image processing code. What I am trying to do is target specific colors in an image and change them to a different RGB values, in order to reduce the total number of colors in the image. The workflow for this process goes as follows:
Identify all colors in an image.
Identify a color to remove.
Round that color to the next nearest color in the image.
Find all pixels in image that match that color and change them to the rounded color.
Repeat above steps until number of colors threshold is met.
I have all of this code functioning in C#, but now I want the pixel changing done on the gpu since running through 250000 pixels multiple times in a row can take a couple of minutes.
using OpenCL.Net;
private Context _context;
private Device _device;
public void SetupOpenCL()
{
ErrorCode error;
Platform[] platforms = Cl.GetPlatformIDs(out error);
List<Device> devicesList = new List<Device>();
CheckErr(error, "Cl.GetPlatformIDs");
foreach (Platform platform in platforms)
{
string platformName = Cl.GetPlatformInfo(platform, PlatformInfo.Name, out error).ToString();
Console.WriteLine("Platform: " + platformName);
CheckErr(error, "Cl.GetPlatformInfo");
//We will be looking only for GPU devices
foreach (Device device in Cl.GetDeviceIDs(platform, DeviceType.Gpu, out error))
{
CheckErr(error, "Cl.GetDeviceIDs");
//Console.WriteLine("Device: " + device.ToString());
devicesList.Add(device);
}
}
if (devicesList.Count <= 0)
{
//Console.WriteLine("No devices found.");
//Set bool used to signify that code processed on CPU only
return;
}
_device = devicesList[0];
if (Cl.GetDeviceInfo(_device, DeviceInfo.ImageSupport, out error).CastTo<Bool>() == Bool.False)
{
//Console.WriteLine("No image support.");
//Set bool used to signify that code processed on CPU only
return;
}
_context = Cl.CreateContext(null, 1, new[] { _device }, ContextNotify, IntPtr.Zero, out error); //Second parameter is amount of devices
CheckErr(error, "Cl.CreateContext");
}
Through Visual Studio, I can see that my computer has a Intel(R) Core(TM) i7-4500U CPU # 1.80GHz and a Intel(R) HD Graphics 4400 (GPU). The code that sets up my kernel is as follows:
public Bitmap ImagingTest2(Bitmap inputBM, int[] oldRGB, int[] newRGB)
{
ErrorCode error;
//Load and compile kernel source code.
string programPath = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location) + "\\ImagingTest2.cl";
//The path to the source file may vary
if (!System.IO.File.Exists(programPath))
{
Console.WriteLine("Program doesn't exist at path " + programPath);
return inputBM;
}
string programSource = System.IO.File.ReadAllText(programPath);
using (OpenCL.Net.Program program = Cl.CreateProgramWithSource(_context, 1, new[] { programSource }, null, out error))
{
CheckErr(error, "Cl.CreateProgramWithSource");
//Compile kernel source
error = Cl.BuildProgram(program, 1, new[] { _device }, string.Empty, null, IntPtr.Zero);
CheckErr(error, "Cl.BuildProgram");
//Check for any compilation errors
if (Cl.GetProgramBuildInfo(program, _device, ProgramBuildInfo.Status, out error).CastTo<BuildStatus>()
!= BuildStatus.Success)
{
CheckErr(error, "Cl.GetProgramBuildInfo");
Console.WriteLine("Cl.GetProgramBuildInfo != Success");
Console.WriteLine(Cl.GetProgramBuildInfo(program, _device, ProgramBuildInfo.Log, out error));
return inputBM;
}
//Create the required kernel (entry function)
Kernel kernel = Cl.CreateKernel(program, "imagingTest", out error);
CheckErr(error, "Cl.CreateKernel");
int intPtrSize = 0;
intPtrSize = Marshal.SizeOf(typeof(IntPtr));
//Image's RGBA data converted to an unmanaged[] array
byte[] inputByteArray;
//OpenCL memory buffer that will keep our image's byte[] data.
Mem inputImage2DBuffer;
OpenCL.Net.ImageFormat clImageFormat = new OpenCL.Net.ImageFormat(ChannelOrder.RGBA, ChannelType.Unsigned_Int8);
int inputImgWidth, inputImgHeight;
int inputImgBytesSize;
int inputImgStride;
inputImgWidth = inputBM.Width;
inputImgHeight = inputBM.Height;
//Get raw pixel data of the bitmap
//The format should match the format of clImageFormat
BitmapData bitmapData = inputBM.LockBits(new Rectangle(0, 0, inputBM.Width, inputBM.Height),
ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);//inputImage.PixelFormat);
inputImgStride = bitmapData.Stride;
inputImgBytesSize = bitmapData.Stride * bitmapData.Height;
//Copy the raw bitmap data to an unmanaged byte[] array
inputByteArray = new byte[inputImgBytesSize];
Marshal.Copy(bitmapData.Scan0, inputByteArray, 0, inputImgBytesSize);
//Allocate OpenCL image memory buffer
inputImage2DBuffer = (Mem)Cl.CreateImage2D(_context, MemFlags.CopyHostPtr | MemFlags.ReadOnly, clImageFormat,
(IntPtr)bitmapData.Width, (IntPtr)bitmapData.Height,
(IntPtr)0, inputByteArray, out error);
CheckErr(error, "Cl.CreateImage2D input");
//Unmanaged output image's raw RGBA byte[] array
byte[] outputByteArray = new byte[inputImgBytesSize];
//Allocate OpenCL image memory buffer
Mem outputImage2DBuffer = (Mem)Cl.CreateImage2D(_context, MemFlags.CopyHostPtr |
MemFlags.WriteOnly, clImageFormat, (IntPtr)inputImgWidth,
(IntPtr)inputImgHeight, (IntPtr)0, outputByteArray, out error);
CheckErr(error, "Cl.CreateImage2D output");
//Pass the memory buffers to our kernel function
error = Cl.SetKernelArg(kernel, 0, (IntPtr)intPtrSize, inputImage2DBuffer);
error |= Cl.SetKernelArg(kernel, 1, (IntPtr)intPtrSize, outputImage2DBuffer);
error = Cl.SetKernelArg(kernel, 2, (IntPtr)(sizeof(int) * 4), oldRGB);
error = Cl.SetKernelArg(kernel, 3, (IntPtr)(sizeof(int) * 4), newRGB);
CheckErr(error, "Cl.SetKernelArg");
//Create a command queue, where all of the commands for execution will be added
CommandQueue cmdQueue = Cl.CreateCommandQueue(_context, _device, (CommandQueueProperties)0, out error);
CheckErr(error, "Cl.CreateCommandQueue");
Event clevent;
//Copy input image from the host to the GPU.
IntPtr[] originPtr = new IntPtr[] { (IntPtr)0, (IntPtr)0, (IntPtr)0 }; //x, y, z
IntPtr[] regionPtr = new IntPtr[] { (IntPtr)inputImgWidth, (IntPtr)inputImgHeight, (IntPtr)1 }; //x, y, z
IntPtr[] workGroupSizePtr = new IntPtr[] { (IntPtr)inputImgWidth, (IntPtr)inputImgHeight, (IntPtr)1 };
error = Cl.EnqueueWriteImage(cmdQueue, inputImage2DBuffer, Bool.True,
originPtr, regionPtr, (IntPtr)0, (IntPtr)0, inputByteArray, 0, null, out clevent);
CheckErr(error, "Cl.EnqueueWriteImage");
//Execute our kernel (OpenCL code)
error = Cl.EnqueueNDRangeKernel(cmdQueue, kernel, 2, null, workGroupSizePtr, null, 0, null, out clevent);
CheckErr(error, "Cl.EnqueueNDRangeKernel");
//Wait for completion of all calculations on the GPU.
error = Cl.Finish(cmdQueue);
CheckErr(error, "Cl.Finish");
//Read the processed image from GPU to raw RGBA data byte[] array
error = Cl.EnqueueReadImage(cmdQueue, outputImage2DBuffer, Bool.True, originPtr, regionPtr,
(IntPtr)0, (IntPtr)0, outputByteArray, 0, null, out clevent);
CheckErr(error, "Cl.clEnqueueReadImage");
//Clean up memory
Cl.ReleaseKernel(kernel);
Cl.ReleaseCommandQueue(cmdQueue);
Cl.ReleaseMemObject(inputImage2DBuffer);
Cl.ReleaseMemObject(outputImage2DBuffer);
//Get a pointer to our unmanaged output byte[] array
GCHandle pinnedOutputArray = GCHandle.Alloc(outputByteArray, GCHandleType.Pinned);
IntPtr outputBmpPointer = pinnedOutputArray.AddrOfPinnedObject();
//Create a new bitmap with processed data and save it to a file.
Bitmap outputBitmap = new Bitmap(inputImgWidth, inputImgHeight,
inputImgStride, PixelFormat.Format32bppArgb, outputBmpPointer);
inputBM.UnlockBits(bitmapData);
pinnedOutputArray.Free();
return outputBitmap;
}
}
Where inputBM is a known Bitmap that I have the RGB data for every pixel. oldRGB is the RGB values of the pixels that I want to change. newRGB is the RGB values that I want to change the pixels to.
Finally, here is the code for the kernel itself:
__kernel void imagingTest(__read_only image2d_t srcImg,__write_only image2d_t dstImg, int4 oldRGB, int4 newRGB)
{
const sampler_t smp = CLK_NORMALIZED_COORDS_FALSE | //Natural coordinates
CLK_ADDRESS_CLAMP_TO_EDGE |
CLK_FILTER_NEAREST;
int2 coord = (int2)(get_global_id(0), get_global_id(1));
int4 bgra = read_imagei(srcImg, smp, coord); //The byte order is BGRA
//printf("%v4i\n", bgra); //For Debug only
if(oldRGB.x == bgra.z && oldRGB.y == bgra.y && oldRGB.z == bgra.x)
{
bgra.x = (int) newRGB.z;
bgra.y = (int) newRGB.y;
bgra.z = (int) newRGB.x;
bgra.w = 255;
}
write_imagei(dstImg, coord, bgra);
}
I can compile and execute this code successfully. The problem is that the image returned from the gpu is very different colorwise. Using the printf command, I found that the RGB values I get for my input image are different in C# when I use Bitmap.GetPixel from when I use read_imagei within the kernel. He is what I got during one of my tests.
RGB values from Bitmap containing 6 colors using Bitmap.GetPixel:
255,52,12
226,105,123
255,206,206
246,167,180
250,250,250
255,213,213
Same Bitmap passed to the gpu returns the following RGB values with read_imagei:
255,0,0
255,192,192
255,120,120
255,160,160
255,255,255
255,220,220
I'm not sure if this difference is being cause by a image formatting issue or possibly by some sort of rounding that the gpu is performing during the read_imagei command. I'm a self taught programmer and I have hit the limits of my googling capabilities. Any explanation for or help with this problem would be appreciated.
Thanks,
******** Update ********
So I ran another test with an image that had a few more colors. When I read the pixels in C# using Bitmap.GetPixel, I found 10 unique colors. When I read the image on the gpu using read_imagei, it returned 17 unique colors. I have no idea why there is this difference in the number of colors.
RGB values from Bitmap containing 10 colors using Bitmap.GetPixel:
217,0,40
250,133,0
255,157,37
254,231,61
7,113,26
28,136,23
1,110,165
121,54,102
96,78,135
247,111,1
Same Bitmap passed to the gpu returns the following RGB values with read_imagei:
228,3,3
255,140,0
247,111,1
255,237,0
255,137,0
255,173,0
255,241,0
255,240,0
0,118,40
0,128,38
0,77,255
0,75,255
0,130,0
0,94,205
121,0,127
117,7,135
56,61,222
I've been trying to copy a transparent PNG image to clipboard and preserve its transparency to paste it into a specific program that supports it.
I tried many solutions already but the background always ended up gray in one way or another.
So I tried copying the same image using Chrome and pasting it into the program and it worked. It preserved transparency. So then I tried Getting the image from the Clipboard that I had copied using Chrome and Set the image again, expecting the transparency to still be there - but no, transparency was not preserved even though I just took the image from the clipboard and set it again.
var img = Clipboard.GetImage(); // copied using Chrome and transparency is preserved
Clipboard.SetImage(img); // transparency lost
Same issue even if I use the System.Windows.Forms.Clipboard or try getting and setting the DataObject instead of the Image.
The Windows clipboard, by default, does not support transparency, but you can put content on the clipboard in many types together to make sure most applications find some type in it that they can use. Sadly, the most common type, DeviceIndependentBitmap (which Windows itself seems to use) is a really dirty and unreliable one. I wrote a big rant explanation about that here.
I'll assume you have read through that before continuing with my answer here, because it contains the background information required for the next part.
Now, the cleanest way of putting an image on the clipboard with transparency support is a PNG stream, but it won't guarantee that all applications can paste it. Gimp supports PNG paste, and apparently so do the newer MS Office programs, but Google Chrome, for example, doesn't, and will only accept the messy DIB type detailed in the answer I linked to. On the other hand, Gimp will not accept DIB as having transparency, because its creators actually followed the format's specifications, and realized that the format was unreliable (as clearly demonstrated by that question I linked).
Because of the DIB mess, sadly, the best thing to do is simply to put it in there in as many generally-supported types as you can, including PNG, DIB and the normal Image.
PNG and DIB are both put on the clipboard in the same way: by putting them in the DataObject as MemoryStream, and then giving the clipboard the "copy" instruction when actually putting it on.
Most of this is straightforward, but the DIB one is a bit more complex. Note that the following part contains a couple of references to my own toolsets. The GetImageData one can be found in this answer, the BuildImage one can be found here, and the ArrayUtils ones are given below.
These toolsets all use System.Drawing, though. You'll have to figure out for yourself exactly how to do the same things in WPF.
/// <summary>
/// Copies the given image to the clipboard as PNG, DIB and standard Bitmap format.
/// </summary>
/// <param name="image">Image to put on the clipboard.</param>
/// <param name="imageNoTr">Optional specifically nontransparent version of the image to put on the clipboard.</param>
/// <param name="data">Clipboard data object to put the image into. Might already contain other stuff. Leave null to create a new one.</param>
public static void SetClipboardImage(Bitmap image, Bitmap imageNoTr, DataObject data)
{
Clipboard.Clear();
if (data == null)
data = new DataObject();
if (imageNoTr == null)
imageNoTr = image;
using (MemoryStream pngMemStream = new MemoryStream())
using (MemoryStream dibMemStream = new MemoryStream())
{
// As standard bitmap, without transparency support
data.SetData(DataFormats.Bitmap, true, imageNoTr);
// As PNG. Gimp will prefer this over the other two.
image.Save(pngMemStream, ImageFormat.Png);
data.SetData("PNG", false, pngMemStream);
// As DIB. This is (wrongly) accepted as ARGB by many applications.
Byte[] dibData = ConvertToDib(image);
dibMemStream.Write(dibData, 0, dibData.Length);
data.SetData(DataFormats.Dib, false, dibMemStream);
// The 'copy=true' argument means the MemoryStreams can be safely disposed after the operation.
Clipboard.SetDataObject(data, true);
}
}
/// <summary>
/// Converts the image to Device Independent Bitmap format of type BITFIELDS.
/// This is (wrongly) accepted by many applications as containing transparency,
/// so I'm abusing it for that.
/// </summary>
/// <param name="image">Image to convert to DIB</param>
/// <returns>The image converted to DIB, in bytes.</returns>
public static Byte[] ConvertToDib(Image image)
{
Byte[] bm32bData;
Int32 width = image.Width;
Int32 height = image.Height;
// Ensure image is 32bppARGB by painting it on a new 32bppARGB image.
using (Bitmap bm32b = new Bitmap(image.Width, image.Height, PixelFormat.Format32bppArgb))
{
using (Graphics gr = Graphics.FromImage(bm32b))
gr.DrawImage(image, new Rectangle(0, 0, bm32b.Width, bm32b.Height));
// Bitmap format has its lines reversed.
bm32b.RotateFlip(RotateFlipType.Rotate180FlipX);
Int32 stride;
bm32bData = ImageUtils.GetImageData(bm32b, out stride);
}
// BITMAPINFOHEADER struct for DIB.
Int32 hdrSize = 0x28;
Byte[] fullImage = new Byte[hdrSize + 12 + bm32bData.Length];
//Int32 biSize;
ArrayUtils.WriteIntToByteArray(fullImage, 0x00, 4, true, (UInt32)hdrSize);
//Int32 biWidth;
ArrayUtils.WriteIntToByteArray(fullImage, 0x04, 4, true, (UInt32)width);
//Int32 biHeight;
ArrayUtils.WriteIntToByteArray(fullImage, 0x08, 4, true, (UInt32)height);
//Int16 biPlanes;
ArrayUtils.WriteIntToByteArray(fullImage, 0x0C, 2, true, 1);
//Int16 biBitCount;
ArrayUtils.WriteIntToByteArray(fullImage, 0x0E, 2, true, 32);
//BITMAPCOMPRESSION biCompression = BITMAPCOMPRESSION.BITFIELDS;
ArrayUtils.WriteIntToByteArray(fullImage, 0x10, 4, true, 3);
//Int32 biSizeImage;
ArrayUtils.WriteIntToByteArray(fullImage, 0x14, 4, true, (UInt32)bm32bData.Length);
// These are all 0. Since .net clears new arrays, don't bother writing them.
//Int32 biXPelsPerMeter = 0;
//Int32 biYPelsPerMeter = 0;
//Int32 biClrUsed = 0;
//Int32 biClrImportant = 0;
// The aforementioned "BITFIELDS": colour masks applied to the Int32 pixel value to get the R, G and B values.
ArrayUtils.WriteIntToByteArray(fullImage, hdrSize + 0, 4, true, 0x00FF0000);
ArrayUtils.WriteIntToByteArray(fullImage, hdrSize + 4, 4, true, 0x0000FF00);
ArrayUtils.WriteIntToByteArray(fullImage, hdrSize + 8, 4, true, 0x000000FF);
Array.Copy(bm32bData, 0, fullImage, hdrSize + 12, bm32bData.Length);
return fullImage;
}
Now, as for getting an image off the clipboard, I noticed there is apparently a difference in behaviour between .Net 3.5 and the later ones, which seem to actually use that DIB. Given that difference, and given how unreliable the DIB format is, you'll want to actually check manually for all types, preferably starting with the completely reliable PNG format.
You can get the DataObject from the clipboard with this code:
DataObject retrievedData = Clipboard.GetDataObject() as DataObject;
The CloneImage function used here is basically just the combination of my GetImageData and BuildImage toolsets, ensuring that a new image is created without any backing resources that might mess up; image objects are known to cause crashes when they're based on a Stream that then gets disposed. A compacted and optimised version of it was posted here, in a question well worth reading on the subject of why this cloning is so important.
/// <summary>
/// Retrieves an image from the given clipboard data object, in the order PNG, DIB, Bitmap, Image object.
/// </summary>
/// <param name="retrievedData">The clipboard data.</param>
/// <returns>The extracted image, or null if no supported image type was found.</returns>
public static Bitmap GetClipboardImage(DataObject retrievedData)
{
Bitmap clipboardimage = null;
// Order: try PNG, move on to try 32-bit ARGB DIB, then try the normal Bitmap and Image types.
if (retrievedData.GetDataPresent("PNG", false))
{
MemoryStream png_stream = retrievedData.GetData("PNG", false) as MemoryStream;
if (png_stream != null)
using (Bitmap bm = new Bitmap(png_stream))
clipboardimage = ImageUtils.CloneImage(bm);
}
if (clipboardimage == null && retrievedData.GetDataPresent(DataFormats.Dib, false))
{
MemoryStream dib = retrievedData.GetData(DataFormats.Dib, false) as MemoryStream;
if (dib != null)
clipboardimage = ImageFromClipboardDib(dib.ToArray());
}
if (clipboardimage == null && retrievedData.GetDataPresent(DataFormats.Bitmap))
clipboardimage = new Bitmap(retrievedData.GetData(DataFormats.Bitmap) as Image);
if (clipboardimage == null && retrievedData.GetDataPresent(typeof(Image)))
clipboardimage = new Bitmap(retrievedData.GetData(typeof(Image)) as Image);
return clipboardimage;
}
public static Bitmap ImageFromClipboardDib(Byte[] dibBytes)
{
if (dibBytes == null || dibBytes.Length < 4)
return null;
try
{
Int32 headerSize = (Int32)ArrayUtils.ReadIntFromByteArray(dibBytes, 0, 4, true);
// Only supporting 40-byte DIB from clipboard
if (headerSize != 40)
return null;
Byte[] header = new Byte[40];
Array.Copy(dibBytes, header, 40);
Int32 imageIndex = headerSize;
Int32 width = (Int32)ArrayUtils.ReadIntFromByteArray(header, 0x04, 4, true);
Int32 height = (Int32)ArrayUtils.ReadIntFromByteArray(header, 0x08, 4, true);
Int16 planes = (Int16)ArrayUtils.ReadIntFromByteArray(header, 0x0C, 2, true);
Int16 bitCount = (Int16)ArrayUtils.ReadIntFromByteArray(header, 0x0E, 2, true);
//Compression: 0 = RGB; 3 = BITFIELDS.
Int32 compression = (Int32)ArrayUtils.ReadIntFromByteArray(header, 0x10, 4, true);
// Not dealing with non-standard formats.
if (planes != 1 || (compression != 0 && compression != 3))
return null;
PixelFormat fmt;
switch (bitCount)
{
case 32:
fmt = PixelFormat.Format32bppRgb;
break;
case 24:
fmt = PixelFormat.Format24bppRgb;
break;
case 16:
fmt = PixelFormat.Format16bppRgb555;
break;
default:
return null;
}
if (compression == 3)
imageIndex += 12;
if (dibBytes.Length < imageIndex)
return null;
Byte[] image = new Byte[dibBytes.Length - imageIndex];
Array.Copy(dibBytes, imageIndex, image, 0, image.Length);
// Classic stride: fit within blocks of 4 bytes.
Int32 stride = (((((bitCount * width) + 7) / 8) + 3) / 4) * 4;
if (compression == 3)
{
UInt32 redMask = ArrayUtils.ReadIntFromByteArray(dibBytes, headerSize + 0, 4, true);
UInt32 greenMask = ArrayUtils.ReadIntFromByteArray(dibBytes, headerSize + 4, 4, true);
UInt32 blueMask = ArrayUtils.ReadIntFromByteArray(dibBytes, headerSize + 8, 4, true);
// Fix for the undocumented use of 32bppARGB disguised as BITFIELDS. Despite lacking an alpha bit field,
// the alpha bytes are still filled in, without any header indication of alpha usage.
// Pure 32-bit RGB: check if a switch to ARGB can be made by checking for non-zero alpha.
// Admitted, this may give a mess if the alpha bits simply aren't cleared, but why the hell wouldn't it use 24bpp then?
if (bitCount == 32 && redMask == 0xFF0000 && greenMask == 0x00FF00 && blueMask == 0x0000FF)
{
// Stride is always a multiple of 4; no need to take it into account for 32bpp.
for (Int32 pix = 3; pix < image.Length; pix += 4)
{
// 0 can mean transparent, but can also mean the alpha isn't filled in, so only check for non-zero alpha,
// which would indicate there is actual data in the alpha bytes.
if (image[pix] == 0)
continue;
fmt = PixelFormat.Format32bppPArgb;
break;
}
}
else
// Could be supported with a system that parses the colour masks,
// but I don't think the clipboard ever uses these anyway.
return null;
}
Bitmap bitmap = ImageUtils.BuildImage(image, width, height, stride, fmt, null, null);
// This is bmp; reverse image lines.
bitmap.RotateFlip(RotateFlipType.Rotate180FlipX);
return bitmap;
}
catch
{
return null;
}
}
Because BitConverter always requires that dumb check on system endianness, I got my own ReadIntFromByteArray and WriteIntToByteArray in an ArrayUtils class:
public static void WriteIntToByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian, UInt32 value)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to write a " + bytes + "-byte value at offset " + startIndex + ".");
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
data[offs] = (Byte)(value >> (8 * index) & 0xFF);
}
}
public static UInt32 ReadIntFromByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to read a " + bytes + "-byte value at offset " + startIndex + ".");
UInt32 value = 0;
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
value += (UInt32)(data[offs] << (8 * index));
}
return value;
}
Im currently trying to use writeablebitmap to take a IntPtr of a scan of images and turn each one into a Bitmap. Im wanting to use writeablebitmap because im having an issue with standard gdi
GDI+ System.Drawing.Bitmap gives error Parameter is not valid intermittently
There is a method on a WriteableBitmap that called WritePixels
http://msdn.microsoft.com/en-us/library/aa346817.aspx
Im not sure what I set for the buffer and the stride every example I find it shows the stride as 0 although that throws an error. When I set the stride to 5 the image appear black. I know this may not be the most efficient code but any help would be appreciated.
//create bitmap header
bmi = new BITMAPINFOHEADER();
//create initial rectangle
Int32Rect rect = new Int32Rect(0, 0, 0, 0);
//create duplicate intptr to use while in global lock
dibhand = dibhandp;
bmpptr = GlobalLock(dibhand);
//get the pixel sizes
pixptr = GetPixelInfo(bmpptr);
//create writeable bitmap
var wbitm = new WriteableBitmap(bmprect.Width, bmprect.Height, 96.0, 96.0, System.Windows.Media.PixelFormats.Bgr32, null);
//draw the image
wbitm.WritePixels(rect, dibhandp, 10, 0);
//convert the writeable bitmap to bitmap
var stream = new MemoryStream();
var encoder = new JpegBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(wbitm));
encoder.Save(stream);
byte[] buffer = stream.GetBuffer();
var bitmap = new System.Drawing.Bitmap(new MemoryStream(buffer));
GlobalUnlock(dibhand);
GlobalFree(dibhand);
GlobalFree(dibhandp);
GlobalFree(bmpptr);
dibhand = IntPtr.Zero;
return bitmap;
An efficient way to work on Bitmaps in C# is to pass temporarily in unsafe mode (I know I don't answer the question exactly but I think the OP did not manage to use Bitmap, so this could be a solution anyway). You just have to lock bits and you're done:
unsafe private void GaussianFilter()
{
// Working images
using (Bitmap newImage = new Bitmap(width, height))
{
// Lock bits for performance reason
BitmapData newImageData = newImage.LockBits(new Rectangle(0, 0, newImage.Width,
newImage.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
byte* pointer = (byte*)newImageData.Scan0;
int offset = newImageData.Stride - newImageData.Width * 4;
// Compute gaussian filter on temp image
for (int j = 0; j < InputData.Height - 1; ++j)
{
for (int 0 = 1; i < InputData.Width - 1; ++i)
{
// You browse 4 bytes per 4 bytes
// The 4 bytes are: B G R A
byte blue = pointer[0];
byte green = pointer[1];
byte red = pointer[2];
byte alpha = pointer[3];
// Your business here by setting pointer[i] = ...
// If you don't use alpha don't forget to set it to 255 else your whole image will be black !!
// Go to next pixels
pointer += 4;
}
// Go to next line: do not forget pixel at last and first column
pointer += offset;
}
// Unlock image
newImage.UnlockBits(newImageData);
newImage.Save("D:\temp\OCR_gray_gaussian.tif");
}
}
This is really much more efficient than SetPixel(i, j), you just have to be careful about pointer limits (and not forget to unlock data when you're done).
Now to answer your question about stride: the stride is the length in bytes of a line, it is a multiple of 4. In my exemple I use the format Format32bppArgb which uses 4 bytes per pixel (R, G, B and alpha), so newImageData.Stride and newImageData.Width * 4 are always the same. I use the offset in my loops only to show where it would be necessary.
But if you use another format, for instance Format24bppRgb which uses 3 bytes per pixel (R, G and B only), then there may be an offset between stride and width. For an image 10 * 10 pixels in this format, you will have a stride of 10 * 3 = 30, + 2 to reach nearest multiple of 4, i.e. 32.