Develop Reference

How can i use LockBits with a Bitmap to scan for white pixels and then write to a new bitmap all the non white pixels?

This is what i did in form1 constructor:
Bitmap bmp2 = new Bitmap(#"e:\result1001.jpg");
CropImageWhiteAreas.ImageTrim(bmp2);
bmp2.Save(#"e:\result1002.jpg");
bmp2.Dispose();
The class CropImageWhiteAreas:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
namespace Test
{
class CropImageWhiteAreas
{
public static Bitmap ImageTrim(Bitmap img)
{
//get image data
BitmapData bd = img.LockBits(new Rectangle(Point.Empty, img.Size),
ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
int[] rgbValues = new int[img.Height * img.Width];
Marshal.Copy(bd.Scan0, rgbValues, 0, rgbValues.Length);
img.UnlockBits(bd);
#region determine bounds
int left = bd.Width;
int top = bd.Height;
int right = 0;
int bottom = 0;
//determine top
for (int i = 0; i < rgbValues.Length; i++)
{
int color = rgbValues[i] & 0xffffff;
if (color != 0xffffff)
{
int r = i / bd.Width;
int c = i % bd.Width;
if (left > c)
{
left = c;
}
if (right < c)
{
right = c;
}
bottom = r;
top = r;
break;
}
}
//determine bottom
for (int i = rgbValues.Length - 1; i >= 0; i--)
{
int color = rgbValues[i] & 0xffffff;
if (color != 0xffffff)
{
int r = i / bd.Width;
int c = i % bd.Width;
if (left > c)
{
left = c;
}
if (right < c)
{
right = c;
}
bottom = r;
break;
}
}
if (bottom > top)
{
for (int r = top + 1; r < bottom; r++)
{
//determine left
for (int c = 0; c < left; c++)
{
int color = rgbValues[r * bd.Width + c] & 0xffffff;
if (color != 0xffffff)
{
if (left > c)
{
left = c;
break;
}
}
}
//determine right
for (int c = bd.Width - 1; c > right; c--)
{
int color = rgbValues[r * bd.Width + c] & 0xffffff;
if (color != 0xffffff)
{
if (right < c)
{
right = c;
break;
}
}
}
}
}
int width = right - left + 1;
int height = bottom - top + 1;
#endregion
//copy image data
int[] imgData = new int[width * height];
for (int r = top; r <= bottom; r++)
{
Array.Copy(rgbValues, r * bd.Width + left, imgData, (r - top) * width, width);
}
//create new image
Bitmap newImage = new Bitmap(width, height, PixelFormat.Format32bppArgb);
BitmapData nbd
= newImage.LockBits(new Rectangle(0, 0, width, height),
ImageLockMode.WriteOnly, PixelFormat.Format32bppArgb);
Marshal.Copy(imgData, 0, nbd.Scan0, imgData.Length);
newImage.UnlockBits(nbd);
return newImage;
}
}
}
I also tried before it Peter solution.
In both the result is(This is a screenshot of my facebook after uploaded the image) still the white areas around:
You can the rectangle around the image i just uploaded and see what i mean by white area around.

If I understand correctly, you have found a sample code snippet that uses LockBits(), but you are not sure how it works or how to modify it to suit your specific need. So I will try to answer from that perspective.
First, a wild guess (since you didn't include the implementation of the LockBitmap class you're using in the first example): the LockBitmap class is some kind of helper class that is supposed to encapsulate the work of calling LockBits() and using the result, including providing versions of GetPixel() and SetPixel() which are presumably much faster than calling those methods on a Bitmap object directly (i.e. access the buffer obtained by calling LockBits()).
If that's the case, then modifying the first example to suit your need is probably best:
public void Change(Bitmap bmp)
{
Bitmap newBitmap = new Bitmap(bmp.Width, bmp.Height, bmp.PixelFormat);
LockBitmap source = new LockBitmap(bmp),
target = new LockBitmap(newBitmap);
source.LockBits();
target.LockBits();
Color white = Color.FromArgb(255, 255, 255, 255);
for (int y = 0; y < source.Height; y++)
{
for (int x = 0; x < source.Width; x++)
{
Color old = source.GetPixel(x, y);
if (old != white)
{
target.SetPixel(x, y, old);
}
}
}
source.UnlockBits();
target.UnlockBits();
newBitmap.Save("d:\\result.png");
}
In short: copy the current pixel value to a local variable, compare that value to the white color value, and if it is not the same, go ahead and copy the pixel value to the new bitmap.
Some variation on the second code example should work as well. The second code example does explicitly what is (I've assumed) encapsulated inside the LockBitmap class that the first code example uses. If for some reason, the first approach isn't suitable for your needs, you can follow the second example.
In that code example you provide, most of the method there is just handling the "grunt work" of locking the bitmap so that the raw data can be accessed, and then iterating through that raw data.
It computes the oRow and nRow array offsets (named for "old row" and "new row", I presume) based on the outer y loop, and then accesses individual pixel data by computing the offset within a given row based on the inner x loop.
Since you want to do essentially the same thing, but instead of converting the image to grayscale, you just want to selectively copy all non-white pixels to the new bitmap, you can (should be able to) simply modify the body of the inner x loop. For example:
byte red = oRow[x * pixelSize + 2],
green = oRow[x * pixelSize + 1],
blue = oRow[x * pixelSize];
if (red != 255 || green != 255 || blue != 255)
{
nRow[x * pixelSize + 2] = red;
nRow[x * pixelSize + 1] = green;
nRow[x * pixelSize] = blue;
}
The above would entirely replace the body of the inner x loop.
One caveat: do note that when using the LockBits() approach, knowing the pixel format of the bitmap is crucial. The example you've shown assumes the bitmaps are in 24 bpp format. If your own bitmaps are in this format, then you don't need to change anything. But if they are in a different format, you'll need to adjust the code to suit that. For example, if your bitmap is in 32 bpp format, you need to pass the correct PixelFormat value to the LockBits() method calls, and then set pixelSize to 4 instead of 3 as the code does now.
Edit:
You've indicated that you would like to crop the new image so that it is the minimize size required to contain all of the non-white pixels. Here is a version of the first example above that should accomplish that:
public void Change(Bitmap bmp)
{
LockBitmap source = new LockBitmap(bmp);
source.LockBits();
Color white = Color.FromArgb(255, 255, 255, 255);
int minX = int.MaxValue, maxX = int.MinValue,
minY = int.MaxValue, maxY = int.MinValue;
// Brute-force scan of the bitmap to find image boundary
for (int y = 0; y < source.Height; y++)
{
for (int x = 0; x < source.Width; x++)
{
if (source.GetPixel(x, y) != white)
{
if (x < minX) minX = x;
if (x > maxX) maxX = x;
if (y < minY) minY = y;
if (y > maxY) maxY = y;
}
}
}
Bitmap newBitmap = new Bitmap(maxX - minx + 1, maxY - minY + 1, bmp.PixelFormat);
LockBitmap target = new LockBitmap(newBitmap);
target.LockBits();
for (int y = 0; y < target.Height; y++)
{
for (int x = 0; x < target.Width; x++)
{
target.SetPixel(x, y, source.GetPixel(x + minX, y + minY));
}
}
source.UnlockBits();
target.UnlockBits();
newBitmap.Save("d:\\result.png");
}
This example includes an initial scan of the original bitmap, after locking it, to find the minimum and maximum coordinate values for any non-white pixel. Having done that, it uses the results of that scan to determine the dimensions of the new bitmap. When copying the pixels, it restricts the x and y loops to the dimensions of the new bitmap, adjusting the x and y values to map from the location in the new bitmap to the given pixel's original location in the old one.
Note that since the initial scan determines where the non-white pixels are, there's no need to check again when actually copying the pixels.
There are more efficient ways to scan the bitmap than the above. This version simply looks at every single pixel in the original bitmap, keeping track of the min and max values for each coordinate. I'm guessing this will be fast enough for your purposes, but if you want something faster, you can change the scan so that it scans for each min and max in sequence:
Scan each row from y of 0 to determine the first row with a non-white pixel. This is the min y value.
Scan each row from y of source.Height - 1 backwards, to find the max y value.
Having found the min and max y values, now scan the columns from x of 0 to find the min x and from source.Width - 1 backwards to find the max x.
Doing it that way involves a lot more code and is probably harder to read and understand, but would involve inspecting many fewer pixels in most cases.
Edit #2:
Here is a sample of the output of the second code example:
Note that all of the white border of the original bitmap (shown on the left side) has been cropped out, leaving only the smallest subset of the original bitmap that can contain all of the non-white pixels (shown on the right side).

Track ball position by image

lets say that i have this image:
http://srv2.jpg.co.il/9/51c614f7c280e.png
i want to get the white ball position(x,y),
this is a verey big image, then i cut the image by rectangle.(because when the image is smaller everything is faster),
the results:
http://srv2.jpg.co.il/1/51c616787a3fa.png
now i want to track the white ball position by his color(white=rbg(255,255,255)),
my code:
Public Function GetBallPosition(ByRef HaxScreenOnly As Bitmap) As Point
For y = 0 To HaxScreenOnly.Height - 1
For x = 0 To HaxScreenOnly.Width - 1
If HaxScreenOnly.GetPixel(x, y) = Color.FromArgb(0, 0, 0) Then
If HaxScreenOnly.GetPixel(x + 8, y) = Color.FromArgb(0, 0, 0) And HaxScreenOnly.GetPixel(x + 8, y + 3) = Color.FromArgb(255, 255, 255) Then
Return New Point(x, y)
End If
End If
Next
Next
Return New Point(0, 0)
End Function
If the color of the current pixel is black and the color of the current pixel(x+8,y+3) is white then this is the ball
it's working...but its verey slow, something like 200 miliseconds to track the ball position.
this is not fast enough.
there is faster way to track the white ball(C# or VB.net)?

Finally, I have you a solution for you. Calling GetPixel is a costly process, but you use Bitmap.LockBits and manipulate / access the image data from a pointer. I took the LockBitmap class from this article.
I checked on the performance from what I was getting previously, which was exactly like you mentioned, around 200ms~.
Here is a picture of the result using LockBitmap, rescanning the image continuously with the optimized code!
static void Main(string[] args)
{
byte[] data = new WebClient().DownloadData("http://srv2.jpg.co.il/1/51c616787a3fa.png");
Image image = Image.FromStream(new MemoryStream(data));
LockBitmap bitmap = new LockBitmap(new Bitmap(image));
// this essentially copies the data into memory and copies from a pointer to an array
bitmap.LockBits();
Color black = Color.FromArgb(0, 0, 0);
Color white = Color.FromArgb(255, 255, 255);
Stopwatch stopwatch = Stopwatch.StartNew();
for (int y = 0; y < bitmap.Height; y++)
{
for (int x = 0; x < bitmap.Width; x++)
{
// GetPixel is a nice abstraction the author in the Article created so we don't have to do any of the gritty stuff.
if (bitmap.GetPixel(x, y) == black)
{
if (bitmap.GetPixel(x + 8, y) == black && bitmap.GetPixel(x + 8, y + 3) == white)
{
Console.WriteLine("White Ball Found in {0}", stopwatch.Elapsed.ToString());
break;
}
}
}
}
bitmap.UnlockBits(); // copies the data from the array back to the original pointer
Console.Read();
}
Hope this helps, it was certainly an interesting read for me.
Update:
As mentioned by King, I was able to further reduce the timing for you based on the algorithm improvement. So we've gone from O(n) to O(n log) time complexity (I think).
for (int y = 0; y < bitmap.Height; y += 3) // As we know the radius of the ball
{
for (int x = 0; x < bitmap.Width; x += 3) // We can increase this
{
if (bitmap.GetPixel(x, y) == black && bitmap.GetPixel(x, y + 3) == white)
{
Console.WriteLine("White Ball Found ({0},{1}) in {2}", x, y, stopwatch.Elapsed.ToString());
break;
}
}
}

Recognizing image within image in C#

I'd like to find an image (needle) within an image (haystack).
To keep things simple I take two screenshots of my desktop. One full size (haystack) and a tiny one (needle). I then loop through the haystack image and try to find the needle image.
capture needle and haystack screenshot
loop through haystack, looking out for haystack[i] == first pixel of needle
[if 2. is true:] loop through the 2nd to last pixel of needle and compare it to haystack[i]
Expected result: the needle image is found at the correct location.
I already got it working for some coordinates/widths/heights (A).
But sometimes bits seem to be "off" and therefore no match is found (B).
What could I be doing wrong? Any suggestions are welcome. Thanks.
var needle_height = 25;
var needle_width = 25;
var haystack_height = 400;
var haystack_width = 500;
A. example input - match
var needle = screenshot(5, 3, needle_width, needle_height);
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);
B. example input - NO match
var needle = screenshot(5, 5, needle_width, needle_height);
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);
1. capture needle and haystack image
private int[] screenshot(int x, int y, int width, int height)
{
Bitmap bmp = new Bitmap(width, height, PixelFormat.Format32bppArgb);
Graphics.FromImage(bmp).CopyFromScreen(x, y, 0, 0, bmp.Size);
var bmd = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly, bmp.PixelFormat);
var ptr = bmd.Scan0;
var bytes = bmd.Stride * bmp.Height / 4;
var result = new int[bytes];
Marshal.Copy(ptr, result, 0, bytes);
bmp.UnlockBits(bmd);
return result;
}
2. try to find a match
public Point findmatch(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int i = 0; i < haystack.Length; i++)
{
if (haystack[i] == firstpixel)
{
var y = i / haystack_height;
var x = i % haystack_width;
var matched = checkmatch(haystack, needle, x, y);
if (matched)
return (new Point(x,y));
}
}
return new Point();
}
3. verify full match
public bool checkmatch(int[] haystack, int[] needle, int startx, int starty)
{
for (int y = starty; y < starty + needle_height; y++)
{
for (int x = startx; x < startx + needle_width; x++)
{
int haystack_index = y * haystack_width + x;
int needle_index = (y - starty) * needle_width + x - startx;
if (haystack[haystack_index] != needle[needle_index])
return false;
}
}
return true;
}

Instead of making two screenshots of your desktop with a time interval between them, I would take a screenshot once and cut "needle" and "haystack" from those same bitmap source. Otherwise you have the risk of a change of your desktop contents between the two moments where the screenshots are taken.
EDIT: And when your problem still occurs after that, I would try to save the image to a file and try again with that file using your debugger, giving you a reproducible situation.

First, there is a problem with the findmatch loop. You shouldn't just use the haystack image as an array, because you need to subtract needle's width and height from right and bottom respectively:
public Point? findmatch(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int y = 0; y < haystack_height - needle_height; y++)
for (int x = 0; x < haystack_width - needle_width; x++)
{
if (haystack[y * haystack_width + x] == firstpixel)
{
var matched = checkmatch(haystack, needle, x, y);
if (matched)
return (new Point(x, y));
}
}
return null;
}
That should probably solve the problem. Also, keep in mind that there might be multiple matches. For example, if "needle" is a completely white rectangle portion of a window, there will most likely be many matches in the entire screen. If this is a possibility, modify your findmatch method to continue searching for results after the first one is found:
public IEnumerable<Point> FindMatches(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int y = 0; y < haystack_height - needle_height; y++)
for (int x = 0; x < haystack_width - needle_width; x++)
{
if (haystack[y * haystack_width + x] == firstpixel)
{
if (checkmatch(haystack, needle, x, y))
yield return (new Point(x, y));
}
}
}
Next, you need to keep a habit of manually disposing all objects which implement IDisposable, which you have created yourself. Bitmap and Graphics are such objects, meaning that your screenshot method needs to be modified to wrap those objects in using statements:
private int[] screenshot(int x, int y, int width, int height)
{
// dispose 'bmp' after use
using (var bmp = new Bitmap(width, height, PixelFormat.Format32bppArgb))
{
// dispose 'g' after use
using (var g = Graphics.FromImage(bmp))
{
g.CopyFromScreen(x, y, 0, 0, bmp.Size);
var bmd = bmp.LockBits(
new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly,
bmp.PixelFormat);
var ptr = bmd.Scan0;
// as David pointed out, "bytes" might be
// a bit misleading name for a length of
// a 32-bit int array (so I've changed it to "len")
var len = bmd.Stride * bmp.Height / 4;
var result = new int[len];
Marshal.Copy(ptr, result, 0, len);
bmp.UnlockBits(bmd);
return result;
}
}
}
The rest of the code seems ok, with the remark that it won't be very efficient for certain inputs. For example, you might have a large solid color as your desktop's background, which might result in many checkmatch calls.
If performance is of interest to you, you might want to check different ways to speed up the search (something like a modified Rabin-Karp comes to mind, but I am sure there are some existing algorithms which ensure that invalid candidates are skipped immediately).

I don't think your equations for haystack_index or needle_index are correct. It looks like you take the Scan0 offset into account when you copy the bitmap data, but you need to use the bitmap's Stride when calculating the byte position.
Also, the Format32bppArgb format uses 4 bytes per pixel. It looks like you are assuming 1 byte per pixel.
Here's the site I used to help with those equations: https://web.archive.org/web/20141229164101/http://bobpowell.net/lockingbits.aspx
Format32BppArgb: Given X and Y coordinates, the address of the first element in the
pixel is Scan0+(y * stride)+(x*4). This Points to the blue byte. The
following three bytes contain the green, red and alpha bytes.

Here are the class reference with example code that works great for my C# application for finding needle in haystack for each frame from a USB camera, in year 2018... I believe Accord is mostly a bunch of C# wrappers for fast C++ code.
Also check out the C# wrapper for Microsoft C++ DirectShow that I use to search for needle within each frame from a USB camera

region growing image segmentation

i make region growing algorithm for my project
this is my algorithm
(my picture have been greyscale before it)
1. get value pixel (0,0) for seed pixel
2. compare value seed pixel with one neighbor pixel
3. if value of no.3 less than treshold (T), go to next pixel and go to no.2
4. if value of no.3 more than treshold (T), change pixel to white(also for next 10 pixel), and get new seed value pixel.
my goal is my picture segmented with white line
this is my code
private void button4_Click(object sender, EventArgs e)
{
// GDI+ still lies to us - the return format is BGR, NOT RGB.
BitmapData bmData = RImage.LockBits(new Rectangle(0, 0, RImage.Width, RImage.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
int stride = bmData.Stride;
System.IntPtr Scan0 = bmData.Scan0;
unsafe
{
byte* p = (byte*)(void*)Scan0;
int nOffset = stride - RImage.Width * 3;
for (int y = 0; y < RImage.Height; ++y)
{
for (int x = 0; x < RImage.Width; ++x)
{
//every new line of x must new seed
if (x == 0)
{
//getting new value seed pixel
seedR = p[x];
seedG = p[x+1];
seedB = p[x+2];
}
//compare value of seed pixel and pixel scan
if ((seedR - p[x] >= tred) || (p[x] - seedR >= tred))
{
//make white line with change value of pixel
for (int i=1; i <= 5; ++i)
{
p[x] = p[x + 1] = p[x + 2] = 0;
x++;
}
//getting new value of seed pixel
seedR = p[x];
seedG = p[x + 1];
seedB = p[x + 2];
}
p += 3;
}
p += nOffset;
}
}
RImage.UnlockBits(bmData);
}
my problem is my image become white in 1/3 of image
what must i doing for "region growing" ??
thx

I've left some questions about your algorithm in the comments, but as I was writing them I realized that what you're trying to do may not be image segmentation at all.
my goal is my picture segmented with white line
Do you mean you want something like this:
If yes, then what you're interested in isn't image segmentation, it's edge detection. If you want to implement something like that, then have a read about convolution as well.

C# Attempted to read or write protected memory error

Addendum: it seems to run correctly when I uncheck "optimize code" which leads me to believe it is some quirky configuration problem
Firstly I am trying to run unmanaged code. I have "allow unsafe code" checked. It is pointing to this line of code here where I am trying to read a bitmap without using the relatively slow getpixel:
byte[] buff = { scanline[xo], scanline[xo + 1], scanline[xo + 2], 0xff };
Entire snippet is below. How can I correct this problem?
private const int PIXELSIZE = 4; // Number of bytes in a pixel
BitmapData mainImageData = mainImage.LockBits(new Rectangle(0, 0, mainImage.Width, mainImage.Height), ImageLockMode.ReadOnly, mainImage.PixelFormat);
List<Point> results = new List<Point>();
foundRects = new List<Rectangle>();
for (int y = 0; y < mainImageData.Height
{
byte* scanline = (byte*)mainImageData.Scan0 + (y * mainImageData.Stride);
for (int x = 0; x < mainImageData.Width; x++)
{
int xo = x * PIXELSIZE;
byte[] buff = { scanline[xo], scanline[xo + 1],
scanline[xo + 2], 0xff };
int val = BitConverter.ToInt32(buff, 0);
// Pixle value from subimage in desktop image
if (pixels.ContainsKey(val) && NotFound(x, y))
{
Point loc = (Point)pixels[val];
int sx = x - loc.X;
int sy = y - loc.Y;
// Subimage occurs in desktop image
if (ImageThere(mainImageData, subImage, sx, sy))
{
Point p = new Point(x - loc.X, y - loc.Y);
results.Add(p);
foundRects.Add(new Rectangle(x, y, subImage.Width,
subImage.Height));
}
}
}

It's hard to say with the limited information we have, but I see a couple of obvious problems, one of which addresses your issue directly:
You are not checking the pixel format, but are assuming that it is 32bppRGB. It is likely 24bppRGB, and that would explain the error.
You are reading the RGB values incorrectly; Windows internally stores bitmaps in BGR order.
You are not calling UnlockBits at the end of the method.

What are the values xo + 1 and xo + 2 when x is mainImageData.Width - 1? They are surely walking off the reservation.

The most likely reason is that the image isn't 32bpp. it could be 24bpp, or possibly even 8 or 16 bit.
You should be getting PIXELSIZE from PixelFormat Rather than hard-coding it to 4.
Check to see that the value you are getting for Stride agrees with the Width.