C# - Capturing the Mouse cursor image - c#

BACKGROUND
I am writing a screen capture application
My code is based derived from this project: http://www.codeproject.com/KB/cs/DesktopCaptureWithMouse.aspx?display=Print
Note that the code captures the the mouse cursor also (which is desirable for me)
MY PROBLEM
Code works fine when the mouse cursor is the normal pointer or hand icon - the mouse is rendered correctly on the screenshot
However, when the mouse cursor is changed to the insertion point (the "I-beam" cursor) - for example typing in NOTEPAD - then code doesn't work - the result is that I get a faint image of the cursor - like a very translucent (gray) version of it instead of the blank & white one would expect.
MY QUESTION
How can I capture the mouse cursor image when the image is one of these "I-beam"-type images
NOTE: If you click on the original article someone offers a suggestion - it doesn't work
SOURCE
This is from the original article.
static Bitmap CaptureCursor(ref int x, ref int y)
{
Bitmap bmp;
IntPtr hicon;
Win32Stuff.CURSORINFO ci = new Win32Stuff.CURSORINFO();
Win32Stuff.ICONINFO icInfo;
ci.cbSize = Marshal.SizeOf(ci);
if (Win32Stuff.GetCursorInfo(out ci))
{
if (ci.flags == Win32Stuff.CURSOR_SHOWING)
{
hicon = Win32Stuff.CopyIcon(ci.hCursor);
if (Win32Stuff.GetIconInfo(hicon, out icInfo))
{
x = ci.ptScreenPos.x - ((int)icInfo.xHotspot);
y = ci.ptScreenPos.y - ((int)icInfo.yHotspot);
Icon ic = Icon.FromHandle(hicon);
bmp = ic.ToBitmap();
return bmp;
}
}
}
return null;
}

While I can't explain exactly why this happens, I think I can show how to get around it.
The ICONINFO struct contains two members, hbmMask and hbmColor, that contain the mask and color bitmaps, respectively, for the cursor (see the MSDN page for ICONINFO for the official documentation).
When you call GetIconInfo() for the default cursor, the ICONINFO struct contains both valid mask and color bitmaps, as shown below (Note: the red border has been added to clearly show the image boundaries):
Default Cursor Mask Bitmap
Default Cursor Color Bitmap
When Windows draws the default cursor, the mask bitmap is first applied with an AND raster operation, then the color bitmap is applied with an XOR raster operation. This results in an opaque cursor and a transparent background.
When you call GetIconInfo() for the I-Beam cursor, though, the ICONINFO struct only contains a valid mask bitmap, and no color bitmap, as shown below (Note: again, the red border has been added to clearly show the image boundaries):
I-Beam Cursor Mask Bitmap
According to the ICONINFO documentation, the I-Beam cursor is then a monochrome cursor. The top half of the mask bitmap is the AND mask, and the bottom half of the mask bitmap is the XOR bitmap. When Windows draws the I-Beam cursor, the top half of this bitmap is first drawn over the desktop with an AND raster operation. The bottom half of the bitmap is then drawn over top with an XOR raster operation. Onscreen, The cursor will appear as the inverse of the content behind it.
One of the comments for the original article that you linked mentions this. On the desktop, since the raster operations are applied over the desktop content, the cursor will appear correct. However, when the image is drawn over no background, as in your posted code, the raster operations that Windows performs result in a faded image.
That being said, this updated CaptureCursor() method will handle both color and monochrome cursors, supplying a plain black cursor image when the cursor is monochrome.
static Bitmap CaptureCursor(ref int x, ref int y)
{
Win32Stuff.CURSORINFO cursorInfo = new Win32Stuff.CURSORINFO();
cursorInfo.cbSize = Marshal.SizeOf(cursorInfo);
if (!Win32Stuff.GetCursorInfo(out cursorInfo))
return null;
if (cursorInfo.flags != Win32Stuff.CURSOR_SHOWING)
return null;
IntPtr hicon = Win32Stuff.CopyIcon(cursorInfo.hCursor);
if (hicon == IntPtr.Zero)
return null;
Win32Stuff.ICONINFO iconInfo;
if (!Win32Stuff.GetIconInfo(hicon, out iconInfo))
return null;
x = cursorInfo.ptScreenPos.x - ((int)iconInfo.xHotspot);
y = cursorInfo.ptScreenPos.y - ((int)iconInfo.yHotspot);
using (Bitmap maskBitmap = Bitmap.FromHbitmap(iconInfo.hbmMask))
{
// Is this a monochrome cursor?
if (maskBitmap.Height == maskBitmap.Width * 2)
{
Bitmap resultBitmap = new Bitmap(maskBitmap.Width, maskBitmap.Width);
Graphics desktopGraphics = Graphics.FromHwnd(Win32Stuff.GetDesktopWindow());
IntPtr desktopHdc = desktopGraphics.GetHdc();
IntPtr maskHdc = Win32Stuff.CreateCompatibleDC(desktopHdc);
IntPtr oldPtr = Win32Stuff.SelectObject(maskHdc, maskBitmap.GetHbitmap());
using (Graphics resultGraphics = Graphics.FromImage(resultBitmap))
{
IntPtr resultHdc = resultGraphics.GetHdc();
// These two operation will result in a black cursor over a white background.
// Later in the code, a call to MakeTransparent() will get rid of the white background.
Win32Stuff.BitBlt(resultHdc, 0, 0, 32, 32, maskHdc, 0, 32, Win32Stuff.TernaryRasterOperations.SRCCOPY);
Win32Stuff.BitBlt(resultHdc, 0, 0, 32, 32, maskHdc, 0, 0, Win32Stuff.TernaryRasterOperations.SRCINVERT);
resultGraphics.ReleaseHdc(resultHdc);
}
IntPtr newPtr = Win32Stuff.SelectObject(maskHdc, oldPtr);
Win32Stuff.DeleteObject(newPtr);
Win32Stuff.DeleteDC(maskHdc);
desktopGraphics.ReleaseHdc(desktopHdc);
// Remove the white background from the BitBlt calls,
// resulting in a black cursor over a transparent background.
resultBitmap.MakeTransparent(Color.White);
return resultBitmap;
}
}
Icon icon = Icon.FromHandle(hicon);
return icon.ToBitmap();
}
There are some issues with the code that may or may not be a problem.
The check for a monochrome cursor simply tests whether the height is twice the width. While this seems logical, the ICONINFO documentation does not mandate that only a monochrome cursor is defined by this.
There is probably a better way to render the cursor that the BitBlt() - BitBlt() - MakeTransparent() combination of method calls I used.

[StructLayout(LayoutKind.Sequential)]
struct CURSORINFO
{
public Int32 cbSize;
public Int32 flags;
public IntPtr hCursor;
public POINTAPI ptScreenPos;
}
[StructLayout(LayoutKind.Sequential)]
struct POINTAPI
{
public int x;
public int y;
}
[DllImport("user32.dll")]
static extern bool GetCursorInfo(out CURSORINFO pci);
[DllImport("user32.dll")]
static extern bool DrawIcon(IntPtr hDC, int X, int Y, IntPtr hIcon);
const Int32 CURSOR_SHOWING = 0x00000001;
public static Bitmap CaptureScreen(bool CaptureMouse)
{
Bitmap result = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height, PixelFormat.Format24bppRgb);
try
{
using (Graphics g = Graphics.FromImage(result))
{
g.CopyFromScreen(0, 0, 0, 0, Screen.PrimaryScreen.Bounds.Size, CopyPixelOperation.SourceCopy);
if (CaptureMouse)
{
CURSORINFO pci;
pci.cbSize = System.Runtime.InteropServices.Marshal.SizeOf(typeof(CURSORINFO));
if (GetCursorInfo(out pci))
{
if (pci.flags == CURSOR_SHOWING)
{
DrawIcon(g.GetHdc(), pci.ptScreenPos.x, pci.ptScreenPos.y, pci.hCursor);
g.ReleaseHdc();
}
}
}
}
}
catch
{
result = null;
}
return result;
}

Here's a modified version of Dimitar's response (using DrawIconEx) that worked for me on multiple screens:
public class ScreenCapturePInvoke
{
[StructLayout(LayoutKind.Sequential)]
private struct CURSORINFO
{
public Int32 cbSize;
public Int32 flags;
public IntPtr hCursor;
public POINTAPI ptScreenPos;
}
[StructLayout(LayoutKind.Sequential)]
private struct POINTAPI
{
public int x;
public int y;
}
[DllImport("user32.dll")]
private static extern bool GetCursorInfo(out CURSORINFO pci);
[DllImport("user32.dll", SetLastError = true)]
static extern bool DrawIconEx(IntPtr hdc, int xLeft, int yTop, IntPtr hIcon, int cxWidth, int cyHeight, int istepIfAniCur, IntPtr hbrFlickerFreeDraw, int diFlags);
private const Int32 CURSOR_SHOWING = 0x0001;
private const Int32 DI_NORMAL = 0x0003;
public static Bitmap CaptureFullScreen(bool captureMouse)
{
var allBounds = Screen.AllScreens.Select(s => s.Bounds).ToArray();
Rectangle bounds = Rectangle.FromLTRB(allBounds.Min(b => b.Left), allBounds.Min(b => b.Top), allBounds.Max(b => b.Right), allBounds.Max(b => b.Bottom));
var bitmap = CaptureScreen(bounds, captureMouse);
return bitmap;
}
public static Bitmap CapturePrimaryScreen(bool captureMouse)
{
Rectangle bounds = Screen.PrimaryScreen.Bounds;
var bitmap = CaptureScreen(bounds, captureMouse);
return bitmap;
}
public static Bitmap CaptureScreen(Rectangle bounds, bool captureMouse)
{
Bitmap result = new Bitmap(bounds.Width, bounds.Height);
try
{
using (Graphics g = Graphics.FromImage(result))
{
g.CopyFromScreen(bounds.Location, Point.Empty, bounds.Size);
if (captureMouse)
{
CURSORINFO pci;
pci.cbSize = Marshal.SizeOf(typeof (CURSORINFO));
if (GetCursorInfo(out pci))
{
if (pci.flags == CURSOR_SHOWING)
{
var hdc = g.GetHdc();
DrawIconEx(hdc, pci.ptScreenPos.x-bounds.X, pci.ptScreenPos.y-bounds.Y, pci.hCursor, 0, 0, 0, IntPtr.Zero, DI_NORMAL);
g.ReleaseHdc();
}
}
}
}
}
catch
{
result = null;
}
return result;
}
}

Based on the other answers I made a version without all the Windows API stuff (for the monochrome part) because the solutions did not work for all monochrome cursors. I create the cursor from the mask by combining the two mask parts.
My solution:
Bitmap CaptureCursor(ref Point position)
{
CURSORINFO cursorInfo = new CURSORINFO();
cursorInfo.cbSize = Marshal.SizeOf(cursorInfo);
if (!GetCursorInfo(out cursorInfo))
return null;
if (cursorInfo.flags != CURSOR_SHOWING)
return null;
IntPtr hicon = CopyIcon(cursorInfo.hCursor);
if (hicon == IntPtr.Zero)
return null;
ICONINFO iconInfo;
if (!GetIconInfo(hicon, out iconInfo))
return null;
position.X = cursorInfo.ptScreenPos.x - iconInfo.xHotspot;
position.Y = cursorInfo.ptScreenPos.y - iconInfo.yHotspot;
using (Bitmap maskBitmap = Bitmap.FromHbitmap(iconInfo.hbmMask))
{
// check for monochrome cursor
if (maskBitmap.Height == maskBitmap.Width * 2)
{
Bitmap cursor = new Bitmap(32, 32, PixelFormat.Format32bppArgb);
Color BLACK = Color.FromArgb(255, 0, 0, 0); //cannot compare Color.Black because of different names
Color WHITE = Color.FromArgb(255, 255, 255, 255); //cannot compare Color.White because of different names
for (int y = 0; y < 32; y++)
{
for (int x = 0; x < 32; x++)
{
Color maskPixel = maskBitmap.GetPixel(x, y);
Color cursorPixel = maskBitmap.GetPixel(x, y + 32);
if (maskPixel == WHITE && cursorPixel == BLACK)
{
cursor.SetPixel(x, y, Color.Transparent);
}
else if (maskPixel == BLACK)
{
cursor.SetPixel(x, y, cursorPixel);
}
else
{
cursor.SetPixel(x, y, cursorPixel == BLACK ? WHITE : BLACK);
}
}
}
return cursor;
}
}
Icon icon = Icon.FromHandle(hicon);
return icon.ToBitmap();
}

This is the patched version with all fixes for the bugs presented on this page:
public static Bitmap CaptureImageCursor(ref Point point)
{
try
{
var cursorInfo = new CursorInfo();
cursorInfo.cbSize = Marshal.SizeOf(cursorInfo);
if (!GetCursorInfo(out cursorInfo))
return null;
if (cursorInfo.flags != CursorShowing)
return null;
var hicon = CopyIcon(cursorInfo.hCursor);
if (hicon == IntPtr.Zero)
return null;
Iconinfo iconInfo;
if (!GetIconInfo(hicon, out iconInfo))
{
DestroyIcon(hicon);
return null;
}
point.X = cursorInfo.ptScreenPos.X - iconInfo.xHotspot;
point.Y = cursorInfo.ptScreenPos.Y - iconInfo.yHotspot;
using (var maskBitmap = Image.FromHbitmap(iconInfo.hbmMask))
{
//Is this a monochrome cursor?
if (maskBitmap.Height == maskBitmap.Width * 2 && iconInfo.hbmColor == IntPtr.Zero)
{
var final = new Bitmap(maskBitmap.Width, maskBitmap.Width);
var hDesktop = GetDesktopWindow();
var dcDesktop = GetWindowDC(hDesktop);
using (var resultGraphics = Graphics.FromImage(final))
{
var resultHdc = resultGraphics.GetHdc();
BitBlt(resultHdc, 0, 0, final.Width, final.Height, dcDesktop, (int)point.X + 3, (int)point.Y + 3, CopyPixelOperation.SourceCopy);
DrawIconEx(resultHdc, 0, 0, cursorInfo.hCursor, 0, 0, 0, IntPtr.Zero, 0x0003);
//TODO: I have to try removing the background of this cursor capture.
//Native.BitBlt(resultHdc, 0, 0, final.Width, final.Height, dcDesktop, (int)point.X + 3, (int)point.Y + 3, Native.CopyPixelOperation.SourceErase);
resultGraphics.ReleaseHdc(resultHdc);
ReleaseDC(hDesktop, dcDesktop);
}
DeleteObject(iconInfo.hbmMask);
DeleteDC(dcDesktop);
DestroyIcon(hicon);
return final;
}
DeleteObject(iconInfo.hbmColor);
DeleteObject(iconInfo.hbmMask);
DestroyIcon(hicon);
}
var icon = Icon.FromHandle(hicon);
return icon.ToBitmap();
}
catch (Exception ex)
{
//You should catch exception with your method here.
//LogWriter.Log(ex, "Impossible to get the cursor.");
}
return null;
}
This version works with:
I-Beam cursors.
Black cursors.
Normal cursors.
Inverted cursors.
See working, here: https://github.com/NickeManarin/ScreenToGif/blob/master/ScreenToGif/Util/Native.cs#L991

Your description of a translucent 'gray' version of the I-beam cursor makes me wonder if you're encountering an issue with image scaling or mispositioning of the cursor.
One of the people posting on that site provided a (broken) link to a report with peculiar behavior that I've tracked down to: http://www.efg2.com/Lab/Graphics/CursorOverlay.htm
The examples on that page are not in C# but the author of the codeproject solution may have been doing something similar and I know I've screwed up my scaling when using the graphics object on plenty of occassions myself:
In any ImageMouseDown event once an
image is loaded, the CusorBitmap is
drawn with transparency on top of the
bitmap using the Canvas.Draw method.
Note some coordinate adjustments
(rescaling) are needed in case the
bitmap is stretched to fit in the
TImage.

Related

How to use PrintDocument with a scrollable Panel?

How do I use PrintDocument with a scrollable panel`?
Here is some of my code:
MemoryImage = new Bitmap(pnl.Width, pnl.Height);
Rectangle rect = new Rectangle(0, 0, pnl.Width, pnl.Height);
pnl.DrawToBitmap(MemoryImage, new Rectangle(0, 0, pnl.Width,
pnl.Height));
Rectangle pagearea = e.PageBounds;
e.Graphics.DrawImage(MemoryImage, (pagearea.Width / 2) -
(pannel.Width / 2), pannel.Location.Y);
These sets of methods allow to print the content of a ScrollableControl to a Bitmap.
A description of the procedure:
The control is first scrolled back to the origin (control.AutoScrollPosition = new Point(0, 0); (an exception is raised otherwise: the Bitmap has a wrong size. You may want to store the current scroll position and restore it after).
Verifies and stores the actual size of the Container, returned by the PreferredSize or DisplayRectangle properties (depending on the conditions set by the method arguments and the type of container printed). This property considers the full extent of a container.
This will be the size of the Bitmap.
Clears the Bitmap using the background color of the Container.
Iterates the ScrollableControl.Controls collection and prints all first-level child controls in their relative position (a child Control's Bounds rectangle is relative to the container ClientArea.)
If a first-level Control has children, calls the DrawNestedControls recursive method, which will enumerate and draw all nested child Containers/Controls, preserving the internal clip bounds.
Includes support for RichTextBox controls.
The RichEditPrinter class contains the logic required to print the content of a RichTextBox/RichEdit control. The class sends an EM_FORMATRANGE message to the RichTextBox, using the Device context of the Bitmap where the control is being printed.
More details available in the MSDN Docs: How to Print the Contents of Rich Edit Controls.
The ScrollableControlToBitmap() method takes only a ScrollableControl type as argument: you cannot pass a TextBox control, even if it uses ScrollBars.
▶ Set the fullSize argument to true or false to include all child controls inside a Container or just those that are visible. If set to true, the Container's ClientRectangle is expanded to include and print all its child Controls.
▶ Set the includeHidden argument to true or false to include or exclude the hidden control, if any.
Note: this code uses the Control.DeviceDpi property to evaluate the current Dpi of the container's Device Context. This property requires .Net Framework 4.7+. If this version is not available, you can remove:
bitmap.SetResolution(canvas.DeviceDpi, canvas.DeviceDpi);
or derive the value with other means. See GetDeviceCaps.
Possibly, update the Project's Framework version :)
// Prints the content of the current Form instance,
// include all child controls and also those that are not visible
var bitmap = ControlPrinter.ScrollableControlToBitmap(this, true, true);
// Prints the content of a ScrollableControl inside a Form
// include all child controls except those that are not visible
var bitmap = ControlPrinter.ScrollableControlToBitmap(this.panel1, true, false);
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
using System.Windows.Forms;
public class ControlPrinter
{
public static Bitmap ScrollableControlToBitmap(ScrollableControl canvas, bool fullSize, bool includeHidden)
{
canvas.AutoScrollPosition = new Point(0, 0);
if (includeHidden) {
canvas.SuspendLayout();
foreach (Control child in canvas.Controls) {
child.Visible = true;
}
canvas.ResumeLayout(true);
}
canvas.PerformLayout();
Size containerSize = canvas.DisplayRectangle.Size;
if (fullSize) {
containerSize.Width = Math.Max(containerSize.Width, canvas.ClientSize.Width);
containerSize.Height = Math.Max(containerSize.Height, canvas.ClientSize.Height);
}
else {
containerSize = canvas.ClientSize;;
}
var bitmap = new Bitmap(containerSize.Width, containerSize.Height, PixelFormat.Format32bppArgb);
bitmap.SetResolution(canvas.DeviceDpi, canvas.DeviceDpi);
var graphics = Graphics.FromImage(bitmap);
if (canvas.BackgroundImage != null) {
graphics.DrawImage(canvas.BackgroundImage, new Rectangle(Point.Empty, containerSize));
}
else {
graphics.Clear(canvas.BackColor);
}
var rtfPrinter = new RichEditPrinter(graphics);
try {
DrawNestedControls(canvas, canvas, new Rectangle(Point.Empty, containerSize), bitmap, rtfPrinter);
return bitmap;
}
finally {
rtfPrinter.Dispose();
graphics.Dispose();
}
}
private static void DrawNestedControls(Control outerContainer, Control parent, Rectangle parentBounds, Bitmap bitmap, RichEditPrinter rtfPrinter)
{
for (int i = parent.Controls.Count - 1; i >= 0; i--) {
var ctl = parent.Controls[i];
if (!ctl.Visible || (ctl.Width < 1 || ctl.Height < 1)) continue;
var clipBounds = Rectangle.Empty;
if (parent.Equals(outerContainer)) { clipBounds = ctl.Bounds; }
else {
Size scrContainerSize = parentBounds.Size;
if ((parent != ctl) && parent is ScrollableControl scrctl) {
if (scrctl.VerticalScroll.Visible) scrContainerSize.Width -= (SystemInformation.VerticalScrollBarWidth + 1);
if (scrctl.HorizontalScroll.Visible) scrContainerSize.Height -= (SystemInformation.HorizontalScrollBarHeight + 1);
}
clipBounds = Rectangle.Intersect(new Rectangle(Point.Empty, scrContainerSize), ctl.Bounds);
}
if (clipBounds.Width < 1 || clipBounds.Height < 1) continue;
var bounds = outerContainer.RectangleToClient(parent.RectangleToScreen(clipBounds));
if (ctl is RichTextBox rtb) {
rtfPrinter.DrawRtf(rtb.Rtf, outerContainer.Bounds, bounds, ctl.BackColor);
}
else {
ctl.DrawToBitmap(bitmap, bounds);
}
if (ctl.HasChildren) {
DrawNestedControls(outerContainer, ctl, clipBounds, bitmap, rtfPrinter);
}
}
}
internal class RichEditPrinter : IDisposable
{
Graphics dc = null;
RTBPrinter rtb = null;
public RichEditPrinter(Graphics graphics)
{
this.dc = graphics;
this.rtb = new RTBPrinter() { ScrollBars = RichTextBoxScrollBars.None };
}
public void DrawRtf(string rtf, Rectangle canvas, Rectangle layoutArea, Color color)
{
rtb.Rtf = rtf;
rtb.Draw(dc, canvas, layoutArea, color);
rtb.Clear();
}
public void Dispose() => this.rtb.Dispose();
private class RTBPrinter : RichTextBox
{
public void Draw(Graphics g, Rectangle hdcArea, Rectangle layoutArea, Color color)
{
using (var brush = new SolidBrush(color)) {
g.FillRectangle(brush, layoutArea);
};
IntPtr hdc = g.GetHdc();
var canvasAreaTwips = new RECT().ToInches(hdcArea);
var layoutAreaTwips = new RECT().ToInches(layoutArea);
var formatRange = new FORMATRANGE() {
charRange = new CHARRANGE() { cpMax = -1, cpMin = 0 },
hdc = hdc,
hdcTarget = hdc,
rect = layoutAreaTwips,
rectPage = canvasAreaTwips
};
IntPtr lParam = Marshal.AllocCoTaskMem(Marshal.SizeOf(formatRange));
Marshal.StructureToPtr(formatRange, lParam, false);
SendMessage(this.Handle, EM_FORMATRANGE, (IntPtr)1, lParam);
Marshal.FreeCoTaskMem(lParam);
g.ReleaseHdc(hdc);
}
[DllImport("User32.dll", CharSet = CharSet.Auto, SetLastError = true)]
internal static extern int SendMessage(IntPtr hWnd, int uMsg, IntPtr wParam, IntPtr lParam);
internal const int WM_USER = 0x0400;
// https://learn.microsoft.com/en-us/windows/win32/controls/em-formatrange
internal const int EM_FORMATRANGE = WM_USER + 57;
[StructLayout(LayoutKind.Sequential)]
internal struct RECT
{
public int Left;
public int Top;
public int Right;
public int Bottom;
public Rectangle ToRectangle() => Rectangle.FromLTRB(Left, Top, Right, Bottom);
public RECT ToInches(Rectangle rectangle)
{
float inch = 14.92f;
return new RECT() {
Left = (int)(rectangle.Left * inch),
Top = (int)(rectangle.Top * inch),
Right = (int)(rectangle.Right * inch),
Bottom = (int)(rectangle.Bottom * inch)
};
}
}
// https://learn.microsoft.com/en-us/windows/win32/api/richedit/ns-richedit-formatrange?
[StructLayout(LayoutKind.Sequential)]
internal struct FORMATRANGE
{
public IntPtr hdcTarget; // A HDC for the target device to format for
public IntPtr hdc; // A HDC for the device to render to, if EM_FORMATRANGE is being used to send the output to a device
public RECT rect; // The area within the rcPage rectangle to render to. Units are measured in twips.
public RECT rectPage; // The entire area of a page on the rendering device. Units are measured in twips.
public CHARRANGE charRange; // The range of characters to format (see CHARRANGE)
}
[StructLayout(LayoutKind.Sequential)]
internal struct CHARRANGE
{
public int cpMin; // First character of range (0 for start of doc)
public int cpMax; // Last character of range (-1 for end of doc)
}
}
}
}
This is how it works:
VB.Net version of the same procedure

How to save drew Graphics of "Paint()" into image using c#?

I actually wanted to Convert RTF into Image so after googling a lot I've got a code that does it by Paint() Event of Picturebox1 and it works perfectly :
private void pictureBox1_Paint(object sender, PaintEventArgs e)
{
e.Graphics.Clear(richTextBox1.BackColor);
e.Graphics.DrawRtfText(this.richTextBox1.Rtf, this.pictureBox1.ClientRectangle);
base.OnPaint(e);
// below code just create an empty image file
Bitmap newBitmap = new Bitmap(pictureBox1.Width, pictureBox1.Height);
e.Graphics.DrawImage(newBitmap, new Rectangle(0, 0, pictureBox1.Width, pictureBox1.Height), new Rectangle(0, 0, pictureBox1.Width, pictureBox1.Height), GraphicsUnit.Pixel);
newBitmap.Save(#"c:\adv.jpg");
}
in the picture above the left is my richTextBox and the right is a Picturebox.
the ISSUE is I don't know how to save Paint() drew graphic into a file because the 3 last lines of my code just save an empty image.
UPDATE #1:
g.SmoothingMode = SmoothingMode.AntiAlias;
g.TextRenderingHint = System.Drawing.Text.TextRenderingHint.ClearTypeGridFit;
g.InterpolationMode = InterpolationMode.HighQualityBicubic;
g.PixelOffsetMode = PixelOffsetMode.HighQuality;
g.Clear(richTextBox1.BackColor);
g.DrawRtfText(this.richTextBox1.Rtf, this.pictureBox1.ClientRectangle);
by changing the graphics from e.graphics to g the issue is resolved but with one other issue that the quality of bitmap is too low. I've Added this bunch of code but I've got same result, the quality is too low!
Any suggestions?
UPDATE #2
here is the Graphics_DrawRtfText class that does the conversion :
public static class Graphics_DrawRtfText
{
private static RichTextBoxDrawer rtfDrawer;
public static void DrawRtfText(this Graphics graphics, string rtf, Rectangle layoutArea)
{
if (Graphics_DrawRtfText.rtfDrawer == null)
{
Graphics_DrawRtfText.rtfDrawer = new RichTextBoxDrawer();
}
Graphics_DrawRtfText.rtfDrawer.Rtf = rtf;
Graphics_DrawRtfText.rtfDrawer.Draw(graphics, layoutArea);
}
private class RichTextBoxDrawer : RichTextBox
{
//Code converted from code found here: http://support.microsoft.com/kb/812425/en-us
//Convert the unit used by the .NET framework (1/100 inch)
//and the unit used by Win32 API calls (twips 1/1440 inch)
private const double anInch = 14.4;
protected override CreateParams CreateParams
{
get
{
CreateParams createParams = base.CreateParams;
if (SafeNativeMethods.LoadLibrary("msftedit.dll") != IntPtr.Zero)
{
createParams.ExStyle |= SafeNativeMethods.WS_EX_TRANSPARENT; // transparent
createParams.ClassName = "RICHEDIT50W";
}
return createParams;
}
}
public void Draw(Graphics graphics, Rectangle layoutArea)
{
//Calculate the area to render.
SafeNativeMethods.RECT rectLayoutArea;
rectLayoutArea.Top = (int)(layoutArea.Top * anInch);
rectLayoutArea.Bottom = (int)(layoutArea.Bottom * anInch);
rectLayoutArea.Left = (int)(layoutArea.Left * anInch);
rectLayoutArea.Right = (int)(layoutArea.Right * anInch);
IntPtr hdc = graphics.GetHdc();
SafeNativeMethods.FORMATRANGE fmtRange;
fmtRange.chrg.cpMax = -1; //Indicate character from to character to
fmtRange.chrg.cpMin = 0;
fmtRange.hdc = hdc; //Use the same DC for measuring and rendering
fmtRange.hdcTarget = hdc; //Point at printer hDC
fmtRange.rc = rectLayoutArea; //Indicate the area on page to print
fmtRange.rcPage = rectLayoutArea; //Indicate size of page
IntPtr wParam = IntPtr.Zero;
wParam = new IntPtr(1);
//Get the pointer to the FORMATRANGE structure in memory
IntPtr lParam = IntPtr.Zero;
lParam = Marshal.AllocCoTaskMem(Marshal.SizeOf(fmtRange));
Marshal.StructureToPtr(fmtRange, lParam, false);
SafeNativeMethods.SendMessage(this.Handle, SafeNativeMethods.EM_FORMATRANGE, wParam, lParam);
//Free the block of memory allocated
Marshal.FreeCoTaskMem(lParam);
//Release the device context handle obtained by a previous call
graphics.ReleaseHdc(hdc);
}
#region SafeNativeMethods
private static class SafeNativeMethods
{
[DllImport("USER32.dll")]
public static extern IntPtr SendMessage(IntPtr hWnd, int msg, IntPtr wp, IntPtr lp);
[DllImport("kernel32.dll", CharSet = CharSet.Auto)]
public static extern IntPtr LoadLibrary(string lpFileName);
[StructLayout(LayoutKind.Sequential)]
public struct RECT
{
public int Left;
public int Top;
public int Right;
public int Bottom;
}
[StructLayout(LayoutKind.Sequential)]
public struct CHARRANGE
{
public int cpMin; //First character of range (0 for start of doc)
public int cpMax; //Last character of range (-1 for end of doc)
}
[StructLayout(LayoutKind.Sequential)]
public struct FORMATRANGE
{
public IntPtr hdc; //Actual DC to draw on
public IntPtr hdcTarget; //Target DC for determining text formatting
public RECT rc; //Region of the DC to draw to (in twips)
public RECT rcPage; //Region of the whole DC (page size) (in twips)
public CHARRANGE chrg; //Range of text to draw (see earlier declaration)
}
public const int WM_USER = 0x0400;
public const int EM_FORMATRANGE = WM_USER + 57;
public const int WS_EX_TRANSPARENT = 0x20;
}
#endregion
}
}
Disclaimer: I don't have the time to dig into the posted extension method but it is interesting and works well, at least when drawing onto a control surface.
But I could reproduce how bad the results are when drawing into a bitmap..
But: When done right the saved results are excellent!
So here here are a few things to keep in mind:
Saving in the Paint event is a bad idea, as this event will be triggered by the system whenever it needs to redraw the control; test by doing a minimize/maximize cycle.
In addition the DrawRtfText semms to create a double-vision effect when drawing into a bitmap.
So make sure you use DrawToBitmap to grab the results. For this you need to place the call to DrawRtfText in the Paint event of a control!
Also make sure to have large enough resolutions both in the control (pixel size) and the Bitmap (dpi) to get nice, crispy and (if needed) printable results.
Do not save to jpg as this is bound to result in blurry text! Png is the format of choice!
Here is a Paint event:
private void panel1_Paint(object sender, PaintEventArgs e)
{
e.Graphics.Clear(richTextBox1.BackColor);
e.Graphics.TextRenderingHint = System.Drawing.Text.TextRenderingHint.AntiAlias;
e.Graphics.SmoothingMode = SmoothingMode.AntiAlias;
Padding pad = new Padding(120, 230, 10, 30); // pick your own numbers!
Size sz = panel1.ClientSize;
Rectangle rect = new Rectangle(pad.Left, pad.Top,
sz.Width - pad.Horizontal, sz.Height - pad.Vertical);
e.Graphics.DrawRtfText(this.richTextBox1.Rtf, rect);
}
Note that it pays to improve on the default quality settings; if you don't the text in the resulting file will break apart when zooming in..
Here is a Save button click:
private void button1_Click(object sender, EventArgs e)
{
Size sz = panel1.ClientSize;
// first we (optionally) create a bitmap in the original panel size:
Rectangle rect1 = panel1.ClientRectangle;
Bitmap bmp = new Bitmap(rect1.Width, rect1.Height);
panel1.DrawToBitmap(bmp, rect);
bmp.Save("D:\\rtfImage1.png", ImageFormat.Png);
// now we create a 4x larger one:
Rectangle rect2 = new Rectangle(0, 0, sz.Width * 4, sz.Height * 4);
Bitmap bmp2 = new Bitmap(rect2.Width, rect2.Height);
// we need to temporarily enlarge the panel:
panel1.ClientSize = rect2.Size;
// now we can let the routine draw
panel1.DrawToBitmap(bmp2, rect2);
// and before saving we optionally can set the dpi resolution
bmp2.SetResolution(300, 300);
// optionally make background transparent:
bmp2.MakeTransparent(richTextBox1.BackColor);
UnSemi(bmp2); // see the link in the comment!
// save text always as png; jpg is only for fotos!
bmp2.Save("D:\\rtfImage2.png", ImageFormat.Png);
// restore the panels size
panel1.ClientSize = sz;
}
I found the result to be really good.
Note that DrawToBitmap will internally trigger the Paint event to grab the drawn graphics.
Of course you don't need both parts - use only the one you want (.e. skip the 1st part, between first and now ) and do use your own numbers. It helps to know what the output shall be and calculate the necessary sizes and resolutions backward from there.
I added the enlarged version because usually the monitor resolution, which is what the controls all have, is rather limited, around 75-100dpi, while print quality starts only at 150dpi..
Here is a link to the UnSemi function
Your code produces an empty image file because you are not drawing anything onto 'newBitmap'.
If you want to draw anything onto 'newBitmap' you need to create a Graphics object from it. As I do not know where 'DrawRtfText' comes from and how it works my guess would be:
private void pictureBox1_Paint(object sender, PaintEventArgs e)
{
//...
Bitmap newBitmap = new Bitmap(pictureBox1.Width, pictureBox1.Height);
Graphics g = Graphics.FromImage(newBitmap);
g.DrawRtfText(this.richTextBox1.Rtf, this.pictureBox1.ClientRectangle);
newBitmap.Save(#"d:\adv.jpg");
}

Bitmap gets mangled when I converted from System.Drawing.Bitmap object to cv::Mat

I have a WPF application that takes a screen shot of the running Handbrake executable using a class called ScreenCapture that I copied from stack overflow.
public class ScreenCapture
{
[DllImport("user32.dll")]
static extern int GetWindowRgn(IntPtr hWnd, IntPtr hRgn);
//Region Flags - The return value specifies the type of the region that the function obtains. It can be one of the following values.
const int ERROR = 0;
const int NULLREGION = 1;
const int SIMPLEREGION = 2;
const int COMPLEXREGION = 3;
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool GetWindowRect(HandleRef hWnd, out RECT lpRect);
[DllImport("gdi32.dll")]
static extern IntPtr CreateRectRgn(int nLeftRect, int nTopRect, int nRightRect, int nBottomRect);
[DllImport("user32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool PrintWindow(IntPtr hwnd, IntPtr hDC, uint nFlags);
[StructLayout(LayoutKind.Sequential)]
public struct RECT
{
public int Left, Top, Right, Bottom;
public RECT(int left, int top, int right, int bottom)
{
Left = left;
Top = top;
Right = right;
Bottom = bottom;
}
public RECT(System.Drawing.Rectangle r) : this(r.Left, r.Top, r.Right, r.Bottom) { }
public int X
{
get { return Left; }
set { Right -= (Left - value); Left = value; }
}
public int Y
{
get { return Top; }
set { Bottom -= (Top - value); Top = value; }
}
public int Height
{
get { return Bottom - Top; }
set { Bottom = value + Top; }
}
public int Width
{
get { return Right - Left; }
set { Right = value + Left; }
}
public System.Drawing.Point Location
{
get { return new System.Drawing.Point(Left, Top); }
set { X = value.X; Y = value.Y; }
}
public System.Drawing.Size Size
{
get { return new System.Drawing.Size(Width, Height); }
set { Width = value.Width; Height = value.Height; }
}
public static implicit operator System.Drawing.Rectangle(RECT r)
{
return new System.Drawing.Rectangle(r.Left, r.Top, r.Width, r.Height);
}
public static implicit operator RECT(System.Drawing.Rectangle r)
{
return new RECT(r);
}
public static bool operator ==(RECT r1, RECT r2)
{
return r1.Equals(r2);
}
public static bool operator !=(RECT r1, RECT r2)
{
return !r1.Equals(r2);
}
public bool Equals(RECT r)
{
return r.Left == Left && r.Top == Top && r.Right == Right && r.Bottom == Bottom;
}
public override bool Equals(object obj)
{
if (obj is RECT)
return Equals((RECT)obj);
else if (obj is System.Drawing.Rectangle)
return Equals(new RECT((System.Drawing.Rectangle)obj));
return false;
}
public override int GetHashCode()
{
return ((System.Drawing.Rectangle)this).GetHashCode();
}
public override string ToString()
{
return string.Format(System.Globalization.CultureInfo.CurrentCulture, "{{Left={0},Top={1},Right={2},Bottom={3}}}", Left, Top, Right, Bottom);
}
}
public Bitmap GetScreenshot(IntPtr ihandle)
{
IntPtr hwnd = ihandle;//handle here
RECT rc;
GetWindowRect(new HandleRef(null, hwnd), out rc);
Bitmap bmp = new Bitmap(rc.Right - rc.Left, rc.Bottom - rc.Top, PixelFormat.Format32bppArgb);
Graphics gfxBmp = Graphics.FromImage(bmp);
IntPtr hdcBitmap;
try
{
hdcBitmap = gfxBmp.GetHdc();
}
catch
{
return null;
}
bool succeeded = PrintWindow(hwnd, hdcBitmap, 0);
gfxBmp.ReleaseHdc(hdcBitmap);
if (!succeeded)
{
gfxBmp.FillRectangle(new SolidBrush(Color.Gray), new Rectangle(Point.Empty, bmp.Size));
}
IntPtr hRgn = CreateRectRgn(0, 0, 0, 0);
GetWindowRgn(hwnd, hRgn);
Region region = Region.FromHrgn(hRgn);//err here once
if (!region.IsEmpty(gfxBmp))
{
gfxBmp.ExcludeClip(region);
gfxBmp.Clear(Color.Transparent);
}
gfxBmp.Dispose();
return bmp;
}
public void WriteBitmapToFile(string filename, Bitmap bitmap)
{
bitmap.Save(filename, ImageFormat.Bmp);
}
So when the button click handler below is called a screenshot of the handbrake window is taken.
I write it to the harddrive to make sure its ok:
handbrake screen shot.
I create an instance of a CLR class library ClassLibrary1::Class1 and call the method "DoSomething" passing it the System.Drawing.Bitmap object.
private void button4_Click(object sender, RoutedEventArgs e)
{
string wName = "HandBrake";
IntPtr hWnd = IntPtr.Zero;
foreach (Process pList in Process.GetProcesses())
{
if (pList.MainWindowTitle.Contains(wName))
{
hWnd = pList.MainWindowHandle;
var sc = new ScreenCapture();
SetForegroundWindow(hWnd);
var bitmap = sc.GetScreenshot(hWnd);
sc.WriteBitmapToFile("handbrake.bmp", bitmap);
Bitmap image1 = (Bitmap)System.Drawing.Image.FromFile("handbrake.bmp", true);
ClassLibrary1.Class1 opencv = new ClassLibrary1.Class1();
opencv.DoSomething(image1);
}
}
}
Inside DoSomething I attempt to convert the System.Drawing.Bitmap to a OpenCV class cv::Mat. I call cv::imwrite to make sure the bitmap is still ok, unfortunately somethings gone wrong: mangled handbrake screenshot
void Class1::DoSomething(Bitmap ^mybitmap)
{
cv::Mat *imgOriginal;
// Lock the bitmap's bits.
Rectangle rect = Rectangle(0, 0, mybitmap->Width, mybitmap->Height);
Imaging::BitmapData^ bmpData = mybitmap->LockBits(rect, Imaging::ImageLockMode::ReadWrite, mybitmap->PixelFormat);
try
{
// Get the address of the first line.
IntPtr ptr = bmpData->Scan0;
// Declare an array to hold the bytes of the bitmap.
// This code is specific to a bitmap with 24 bits per pixels.
int bytes = Math::Abs(bmpData->Stride) * mybitmap->Height;
array<Byte>^rgbValues = gcnew array<Byte>(bytes);
// Copy the RGB values into the array.
System::Runtime::InteropServices::Marshal::Copy(ptr, rgbValues, 0, bytes);
imgOriginal = new cv::Mat(mybitmap->Height, mybitmap->Width, CV_8UC3, (void *)ptr, std::abs(bmpData->Stride));
}
finally { mybitmap->UnlockBits(bmpData); }//Remember to unlock!!!
cv::imwrite("from_mat.bmp", *imgOriginal);
}
Can anybody spot my error?
Since your image is stretched horizontally, I'm betting that you have the wrong pixel format selected. (It's not stretched vertically, nor skewed diagonally, so the stride is correct.) CV_8UC3 specifies 24 bits per pixel, but I think that your BMP file is using 32 bits per pixel.
Switch your pixel format to CV_8UC4, or better yet, read the number of bits per pixel from the image and select the correct CV format based on that.
Side note: Since you're doing sc.WriteBitmapToFile() followed by opencv.DoSomething(Image.FromFile(), the entire bit about how you're capturing the screenshot is irrelevant. You're reading the bitmap from a file; that's all that matters.

Automating mouse clicks on screen

I am looking for a way of creating a program that will perform a mouse click where it finds a certain color on the screen.
For example if there is a red box on the screen, I would want the program to click on the red box in the center of it.
How could I accomplish this in C#?
As you only wanted a general way, I didn't really make it perfect, but here is the idea:
Have a method for taking a screen shot:
public Bitmap ScreenShot()
{
var screenShot = new Bitmap(Screen.PrimaryScreen.Bounds.Width,
Screen.PrimaryScreen.Bounds.Height,
PixelFormat.Format32bppArgb);
using (var g = Graphics.FromImage(screenShot))
{
g.CopyFromScreen(0, 0, 0, 0, Screen.PrimaryScreen.Bounds.Size);
}
return screenShot;
}
And a method to find a certain color in a bitmap:
Note that this implementation can be DRASTICALLY improved using unsafe code and LockBits (read here and here).
public Point? GetFirstPixel(Bitmap bitmap, Color color)
{
for (var y = 0; y < bitmap.Height; y++)
{
for (var x = 0; x < bitmap.Width; x++)
{
if (bitmap.GetPixel(x, y).Equals(color))
{
return new Point(x, y);
}
}
}
return null;
}
Another method you'll need is one for clicking a certain point:
[DllImport("user32.dll",
CharSet=CharSet.Auto,
CallingConvention=CallingConvention.StdCall)]
private static extern void mouse_event(long dwFlags,
long dx,
long dy,
long cButtons,
long dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
public void Click(Point pt)
{
Cursor.Position = pt;
mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, pt.X, pt.Y, 0, 0);
}
And finally, one to wrap it all up:
public bool ClickOnFirstPixel(Color color)
{
var pt = GetFirstPixel(ScreenShot(), color);
if (pt.HasValue)
{
Click(pt.Value);
}
// return whether found pixel and clicked it
return pt.HasValue;
}
Then, the usage would be:
if (ClickOnFirstPixel(Color.Red))
{
Console.WriteLine("Found a red pixel and clicked it!");
}
else
{
Console.WriteLine("Didn't find a red pixel, didn't click.");
}

How would I go about taking a snapshot of a portion of my application's window?

I'm currently attempting to take snapshot of a specified portion of my application's window from a specified starting coordinate (which is where my problem comes in).
Rectangle bounds = new Rectangle((this.Width/2)-400,(this.Height/2)-200, 800,400);
using (Bitmap bitmap = new Bitmap(bounds.Width, bounds.Height, PixelFormat.Format32bppArgb))
{
using (Graphics graphics = Graphics.FromImage(bitmap))
{
IntPtr hdc = graphics.GetHdc();
PrintWindow(this.axS.Handle, hdc, 0);
graphics.ReleaseHdc(hdc);
graphics.Flush();
string file = "example.png";
bitmap.Save(file, ImageFormat.Png);
}
}
I'm attempting to make a dynamic-adaptive method to take a screenshot of the center of the window, even after being resized. I'm not sure how to apply x and y to the screenshot as a starting point for the screenshot. Dimensions will always remain 800,400 and always taking a screenshot of the center of the application regardless of window size.
Every attempt I have pegged, the bitmap took a screenshot from 0 (+800), 0 (+400) where 0, 0 I need to change.
Is Bitmap capable of this? If not, what other method could I use?
You can use SetViewportOrgEx to set the origin on the HDC. I found that the title bar of the window was throwing off the calculation of the center point, so I took that into account as well.
int x = (this.Width / 2) - 400;
int y = ((this.Height + SystemInformation.CaptionHeight) / 2) - 200;
Rectangle bounds = new Rectangle(x, y, 800, 400);
using (Bitmap bitmap = new Bitmap(bounds.Width, bounds.Height, PixelFormat.Format32bppArgb))
{
using (Graphics graphics = Graphics.FromImage(bitmap))
{
IntPtr hdc = graphics.GetHdc();
POINT pt;
SetViewportOrgEx(hdc, -x, -y, out pt);
// rest as before
}
}
And the signatures for SetViewportOrgEx and POINT:
[DllImport("gdi32.dll")]
static extern bool SetViewportOrgEx(IntPtr hdc, int X, int Y, out POINT lpPoint);
[StructLayout(LayoutKind.Sequential)]
public struct POINT
{
public int X;
public int Y;
public POINT(int x, int y)
{
this.X = x;
this.Y = y;
}
public static implicit operator System.Drawing.Point(POINT p)
{
return new System.Drawing.Point(p.X, p.Y);
}
public static implicit operator POINT(System.Drawing.Point p)
{
return new POINT(p.X, p.Y);
}
}
Instead of using PrintWindow try using Graphics.CopyFromScreen which allows you to specify both an upper-left corner as well as dimensions.
http://msdn.microsoft.com/en-us/library/6yfzc507.aspx
Performs a bit-block transfer of color data, corresponding to a rectangle of pixels, from the screen to the drawing surface of the Graphics.
CopyFromScreen works on screen coordinates so you'll have to calculate that for the call.

Categories

Resources