Develop Reference

c#, wpf BitmapImage CopyPixels (8-bit bmp files) sometimes correct, sometimes not

I simply want to convert a previously loaded BMP-File into a byte[][]. It works pretty well for my own testing images (just some black spots on white background) which are all in the 8 bits per pixel format.
Now I tried the same code for some bitmaps somebody gave me ( also black squares, rectangles on white background) but it's not working as I expected it:
I expected a white pixel to be represented by a value of 255 (and black just by 0 ) in the resulting array, but i found different values there. In one case pixels that are supposed to be white end up with a value of 1 in the array.
Again, all these files are of 8 bit color depth.
Also, I noticed, when I open the Images in paint, save them again as a 256 color bitmap, then it works.
So my questions are:
What is causing this problem ? (Do color palettes maybe play a role ?)
And how can I make it work ?
Here's my amateurish code:
public byte[][] ConvertImageToArray (BitmapSource Image)
{
byte[][] Result = null;
if (Image != null)
{
int Index = 0;
int size = Image.PixelWidth * Image.PixelHeight * Image.Format.BitsPerPixel/8;
byte[] RawImg = new byte[size];
BitmapPalette test = Image.Palette;
int stride = (Image.PixelWidth * Image.Format.BitsPerPixel) / 8;
Image.CopyPixels(RawImg, stride, 0);
Result = new byte[Image.PixelHeight][];
int Width = Image.PixelWidth;
for (int i = 0; i < Result.Length; i++)
{
Result[i] = new byte[Width];
for (int k = 0; k < Result[i].Length; k++)
{
Result[i][k] = RawImg[Index++];
}
}
}
return Result;
}

How to convert a colored image to a image that has only two predefined colors?

I am trying to convert a colored image to a image that only has two colors. My approach was first converting the image to a black and white image by using Aforge.Net Threshold class and then convert the black and white pixels into colors that I want. The display is on real-time so this approach introduces a significant delay. I was wondering if there's a more straightforward way of doing this.
Bitmap image = (Bitmap)eventArgs.Frame.Clone();
Grayscale greyscale = new Grayscale(0.2125, 0.7154, 0.0721);
Bitmap grayImage = greyscale.Apply(image);
Threshold threshold = new Threshold(trigger);
threshold.ApplyInPlace(grayImage);
Bitmap colorImage = CreateNonIndexedImage(grayImage);
if (colorFilter)
{
for (int y = 0; y < colorImage.Height; y++)
{
for (int x = 0; x < colorImage.Width; x++)
{
if (colorImage.GetPixel(x, y).R == 0 && colorImage.GetPixel(x, y).G == 0 && colorImage.GetPixel(x, y).B == 0)
{
colorImage.SetPixel(x, y, Color.Blue);
}
else
{
colorImage.SetPixel(x, y, Color.Yellow);
}
}
}
}
private Bitmap CreateNonIndexedImage(Image src)
{
Bitmap newBmp = new Bitmap(src.Width, src.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
using (Graphics gfx = Graphics.FromImage(newBmp))
{
gfx.DrawImage(src, 0, 0);
}
return newBmp;
}

The normal way to match an image to specific colours is to use Pythagorean distance between the colours in a 3D environment with R, G and B as axes. I got a bunch of toolsets for manipulating images and colours, and I'm not too familiar with any external frameworks, so I'll just dig through my stuff and give you the relevant functions.
First of all, the colour replacement itself. This code will match any colour you give to the closest available colour on a limited palette, and return the index in the given array. Note that I left out the "take the square root" part of the Pythagorean distance calculation; we don't need to know the actual distance, we only need to compare them, and that works just as well without that rather CPU-heavy operation.
public static Int32 GetClosestPaletteIndexMatch(Color col, Color[] colorPalette)
{
Int32 colorMatch = 0;
Int32 leastDistance = Int32.MaxValue;
Int32 red = col.R;
Int32 green = col.G;
Int32 blue = col.B;
for (Int32 i = 0; i < colorPalette.Length; i++)
{
Color paletteColor = colorPalette[i];
Int32 redDistance = paletteColor.R - red;
Int32 greenDistance = paletteColor.G - green;
Int32 blueDistance = paletteColor.B - blue;
Int32 distance = (redDistance * redDistance) + (greenDistance * greenDistance) + (blueDistance * blueDistance);
if (distance >= leastDistance)
continue;
colorMatch = i;
leastDistance = distance;
if (distance == 0)
return i;
}
return colorMatch;
}
Now, on a high-coloured image, this palette matching would have to be done for every pixel on the image, but if your input is guaranteed to be paletted already, then you can just do it on the colour palette, reducing your palette lookups to just 256 per image:
Color[] colors = new Color[] {Color.Black, Color.White };
ColorPalette pal = image.Palette;
for(Int32 i = 0; i < pal.Entries.Length; i++)
{
Int32 foundIndex = ColorUtils.GetClosestPaletteIndexMatch(pal.Entries[i], colors);
pal.Entries[i] = colors[foundIndex];
}
image.Palette = pal;
And that's it; all colours on the palette replaced by their closest match.
Note that the Palette property actually makes a new ColorPalette object, and doesn't reference the one in the image, so the code image.Palette.Entries[0] = Color.Blue; would not work, since it'd just modify that unreferenced copy. Because of that, the palette object always has to be taken out, edited and then reassigned to the image.
If you need to save the result to the same filename, there's a trick with a stream you can use, but if you simply need the object to have its palette changed to these two colours, that's really it.
In case you are not sure of the original image format, the process is quite a bit more involved:
As mentioned before in the comments, GetPixel and SetPixel are extremely slow, and it's much more efficient to access the image's underlying bytes. However, unless you are 100% certain what your input type's pixel format is, you can't just go and access these bytes, since you need to know how to read them. A simple workaround for this is to just let the framework do the work for you, by painting your existing image on a new 32 bits per pixel image:
public static Bitmap PaintOn32bpp(Image image, Color? transparencyFillColor)
{
Bitmap bp = new Bitmap(image.Width, image.Height, PixelFormat.Format32bppArgb);
using (Graphics gr = Graphics.FromImage(bp))
{
if (transparencyFillColor.HasValue)
using (System.Drawing.SolidBrush myBrush = new System.Drawing.SolidBrush(Color.FromArgb(255, transparencyFillColor.Value)))
gr.FillRectangle(myBrush, new Rectangle(0, 0, image.Width, image.Height));
gr.DrawImage(image, new Rectangle(0, 0, bp.Width, bp.Height));
}
return bp;
}
Now, you probably want to make sure transparent pixels don't end up as whatever colour happens to be hiding behind an alpha value of 0, so you better specify the transparencyFillColor in this function to give a backdrop to remove any transparency from the source image.
Now we got the high-colour image, the next step is going over the image bytes, converting them to ARGB colours, and matching those to the palette, using the function I gave before. I'd advise making an 8-bit image because they're the easiest to edit as bytes, and the fact they have a colour palette makes it ridiculously easy to replace colours on them after they're created.
Anyway, the bytes. It's probably more efficient for large files to iterate through the bytes in unsafe memory right away, but I generally prefer copying them out. Your choice, of course; if you think it's worth it, you can combine the two functions below to access it directly. Here's a good example for accessing the colour bytes directly.
/// <summary>
/// Gets the raw bytes from an image.
/// </summary>
/// <param name="sourceImage">The image to get the bytes from.</param>
/// <param name="stride">Stride of the retrieved image data.</param>
/// <returns>The raw bytes of the image</returns>
public static Byte[] GetImageData(Bitmap sourceImage, out Int32 stride)
{
BitmapData sourceData = sourceImage.LockBits(new Rectangle(0, 0, sourceImage.Width, sourceImage.Height), ImageLockMode.ReadOnly, sourceImage.PixelFormat);
stride = sourceData.Stride;
Byte[] data = new Byte[stride * sourceImage.Height];
Marshal.Copy(sourceData.Scan0, data, 0, data.Length);
sourceImage.UnlockBits(sourceData);
return data;
}
Now, all you need to do is make an array to represent your 8-bit image, iterate over all bytes per four, and match the colours you get to the ones in your palette. Note that you can never assume that the actual byte length of one line of pixels (the stride) equals the width multiplied by the bytes per pixel. Because of this, while the code does simply add the pixel size to the read offset to get the next pixel on one line, it uses the stride for skipping over whole lines of pixels in the data.
public static Byte[] Convert32BitTo8Bit(Byte[] imageData, Int32 width, Int32 height, Color[] palette, ref Int32 stride)
{
if (stride < width * 4)
throw new ArgumentException("Stride is smaller than one pixel line!", "stride");
Byte[] newImageData = new Byte[width * height];
for (Int32 y = 0; y < height; y++)
{
Int32 inputOffs = y * stride;
Int32 outputOffs = y * width;
for (Int32 x = 0; x < width; x++)
{
// 32bppArgb: Order of the bytes is Alpha, Red, Green, Blue, but
// since this is actually in the full 4-byte value read from the offset,
// and this value is considered little-endian, they are actually in the
// order BGRA. Since we're converting to a palette we ignore the alpha
// one and just give RGB.
Color c = Color.FromArgb(imageData[inputOffs + 2], imageData[inputOffs + 1], imageData[inputOffs]);
// Match to palette index
newImageData[outputOffs] = (Byte)ColorUtils.GetClosestPaletteIndexMatch(c, palette);
inputOffs += 4;
outputOffs++;
}
}
stride = width;
return newImageData;
}
Now we got our 8-bit array. To convert that array to an image you can use the BuildImage function I already posted on another answer.
So finally, using these tools, the conversion code should be something like this:
public static Bitmap ConvertToColors(Bitmap image, Color[] colors)
{
Int32 width = image.Width;
Int32 height = image.Height;
Int32 stride;
Byte[] hiColData;
// use "using" to properly dispose of temporary image object.
using (Bitmap hiColImage = PaintOn32bpp(image, colors[0]))
hiColData = GetImageData(hiColImage, out stride);
Byte[] eightBitData = Convert32BitTo8Bit(hiColData, width, height, colors, ref stride);
return BuildImage(eightBitData, width, height, stride, PixelFormat.Format8bppIndexed, colors, Color.Black);
}
There we go; your image is converted to 8-bit paletted image, for whatever palette you want.
If you want to actually match to black and white and then replace the colours, that's no problem either; just do the conversion with a palette containing only black and white, then take the resulting bitmap's Palette object, replace the colours in it, and assign it back to the image.
Color[] colors = new Color[] {Color.Black, Color.White };
Bitmap newImage = ConvertToColors(image, colors);
ColorPalette pal = newImage.Palette;
pal.Entries[0] = Color.Blue;
pal.Entries[1] = Color.Yellow;
newImage.Palette = pal;

How can I identify the color of the letters in these images?

I am using this article to solve captchas. It works by removing the background from the image using AForge, and then applying Tesseract OCR to the resulting cleaned image.
The problem is, it currently relies on the letters being black, and since each captcha has a different text color, I need to either pass the color to the image cleaner, or change the color of the letters to black. To do either one, I need to know what the existing color of the letters is.
How might I go about identifying the color of the letters?

Using the answer by #Robert Harvey♦ I went and developed the same code using LockBits and unsafe methods to improve it's speed. You must compile with the "Allow unsafe code" flag on. Note that the order of pixels returned from the image is in the bgr not rgb format and I am locking the bitmap using a format of Format24bppRgb to force it to use 3 bytes per colour.
public unsafe Color GetTextColour(Bitmap bitmap)
{
BitmapData bitmapData = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height), ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
try
{
const int bytesPerPixel = 3;
const int red = 2;
const int green = 1;
int halfHeight = bitmap.Height / 2;
byte* row = (byte*)_bitmapData.Scan0 + (halfHeight * _bitmapData.Stride);
Color startingColour = Color.FromArgb(row[red], row[green], row[0]);
for (int wi = bytesPerPixel, wc = _bitmapData.Width * bytesPerPixel; wi < wc; wi += bytesPerPixel)
{
Color thisColour = Color.FromArgb(row[wi + red], row[wi + green], row[wi]);
if (thisColour != startingColour)
{
return thisColour;
}
}
return Color.Empty; //Or some other default value
}
finally
{
bitmap.UnlockBits(bitmapData);
}
}

The solution to this particular problem turned out to be relatively simple. All I had to do is get the color of the edge pixel halfway down the left side of the image, scan pixels to the right until the color changes, and that's the color of the first letter.
public Color GetTextColor(Bitmap bitmap)
{
var y = bitmap.Height/2;
var startingColor = bitmap.GetPixel(0, y);
for (int x = 1; x < bitmap.Width; x++)
{
var thisColor = bitmap.GetPixel(x, y);
if (thisColor != startingColor)
return thisColor;
}
return null;
}

Adjusting the physical dimensions of a Bitmap object

I have a script that returns a heatmap based on a List of Color objects (they're RGB values derived from a Gradient component in a graphical "coding" software called Grasshopper), which looks like this:
Below is an excerpt of my C# heatmap-drawing method that returns a Bitmap.
private Bitmap DrawHeatmap(List<Color> colors, int U, int V){
colorHeatmapArray = new Color[colors.Count()];
for(int i = 0; i < colors.Count(); i++){
colorHeatmapArray[i] = colors[i];
}
// Create heatmap image.
Bitmap map = new Bitmap(U, V, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
int x = 0;
int y = 0;
for(int i = 0; i < colors.Count(); i++){
Color color = colorHeatmapArray[i];
map.SetPixel(x, y, color);
y++;
if (y >= map.Height){
y = 0;
x++;
}
if (x >= map.Width){
break;
}
}
return map;
}
The method I used to save the image is like this:
private void saveBMP(){
_heatmap.Save(Path); // Path is just a string declared somewhere
}
_heatmap is an instance variable, declared like this: private Bitmap _heatmap;, where I stored the Bitmap object, using the DrawHeatmap() method.
The way I displayed the image on the "canvas" of Grasshopper relies on some Grasshopper-specific method, specifically, this snippet
RectangleF rec = Component.Attributes.Bounds;
rec.X = rec.Right + 10;
rec.Height = Height;
rec.Width = Width;
canvas.Graphics.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.NearestNeighbor;
canvas.Graphics.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.Half;
canvas.Graphics.DrawImage(_heatmap, GH_Convert.ToRectangle(rec));
canvas.Graphics.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
canvas.Graphics.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.Default;
canvas.Graphics.DrawRectangle(Pens.Black, GH_Convert.ToRectangle(rec));
However, when I save the Bitmap object, the result I get is a slightly taller version of what I have on the canvas, which looks like this:
Doesn't look very pretty does it?
My question is - on calling the saveBMP() method, is there a way to manipulate the Bitmap to adjust the dimensions so it looks remotely like what I have on the canvas?

Assuming the _heatmap is set from the output of the DrawHeatmap method, then its size should be being set at the point of initialisation in that method to U by V pixels. Once it's saved, verify from the saved file what the size of the output file (ie are it's dimensions as expected given the value of U and V that enter DrawHeatmap?
When you are drawing to the rectangle in the latter code section, are you using the same Height and Width values as earlier?

After some Googling it looks like I found a solution from this link
Specifically:

Algorithm for finding a painted region on a canvas

Update: I am attempting to pull a little clutter out of this post and sum it up more concisely. Please see the original edit if needed.
I am currently attempting to trace a series of single colored blobs on a Bitmap canvas.
e.g. An example of the bitmap I am attempting to trace would look like the following:
alt text http://www.refuctored.com/polygons.bmp
After successfully tracing the outlines of the 3 blobs on the image, I would have a class that held the color of a blob tied to a point list representing the outline of the blob (not all the pixels inside of the blobs).
The problem I am running into is logic in instances where a neighboring pixel has no surrounding pixels other than the previous pixel.
e.g The top example would trace fine, but the second would fail because the pixel has no where to go since the previous pixels have already been used.
alt text http://www.refuctored.com/error.jpg
I am tracing left-to-right, top-to-bottom, favoring diagonal angles over right angles. I must be able to redraw an exact copy of the region based off the data I extract, so the pixels in the list must be in the right order for the copy to work.
Thus far, my attempt has been riddled with failure, and a couple days of pulling my hair out trying to rewrite the algorithms a little different each time to solve the issue. Thus far I have been unsuccessful. Has anyone else had a similar issue like mine who has a good algorithm to find the edges?

One simple trick to avoiding these cul-de-sacs is to double the size of the image you want to trace using a nearest neighbor scaling algorithm before tracing it. Like that you will never get single strips.
The alternative is to use a marching squares algorithm - but it seems to still have one or two cases where it fails: http://www.sakri.net/blog/2009/05/28/detecting-edge-pixels-with-marching-squares-algorithm/

Have you looked at blob detection algorithms? For example, http://opencv.willowgarage.com/wiki/cvBlobsLib if you can integrate OpenCV into your application. Coupled with thresholding to create binary images for each color (or color range) in your image, you could easily find the blobs that are the same color. Repeat for each color in the image, and you have a list of blobs sorted by color.
If you cannot use OpenCV directly, perhaps the paper referenced by that library ("A linear-time component labeling algorithm using contour tracing technique", F.Chang et al.) would provide a good method for finding blobs.

Rather than using recursion, use a stack.
Pseudo-code:
Add initial pixel to polygon
Add initial pixel to stack
while(stack is not empty) {
pop pixel off the stack
foreach (neighbor n of popped pixel) {
if (n is close enough in color to initial pixel) {
Add n to polygon
Add n to stack
}
}
}
This will use a lot less memory than the same solution using recursion.

Just send your 'Image' to BuildPixelArray function and then call the FindRegions.
After that the 'colors' variable will be holding your colors list and pixel coordinates in every list member.
I've copied the source from one of my projects, there may be some undefined variables or syntax erors.
public class ImageProcessing{
private int[,] pixelArray;
private int imageWidth;
private int imageHeight;
List<MyColor> colors;
public void BuildPixelArray(ref Image myImage)
{
imageHeight = myImage.Height;
imageWidth = myImage.Width;
pixelArray = new int[imageWidth, imageHeight];
Rectangle rect = new Rectangle(0, 0, myImage.Width, myImage.Height);
Bitmap temp = new Bitmap(myImage);
BitmapData bmpData = temp.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
int remain = bmpData.Stride - bmpData.Width * 3;
unsafe
{
byte* ptr = (byte*)bmpData.Scan0;
for (int j = 15; j < bmpData.Height; j++)
{
for (int i = 0; i < bmpData.Width; i++)
{
pixelArray[i, j] = ptr[0] + ptr[1] * 256 + ptr[2] * 256 * 256;
ptr += 3;
}
ptr += remain;
}
}
temp.UnlockBits(bmpData);
}
public void FindRegions()
{
colors = new List<MyColor>();
for (int i = 0; i < imageWidth; i++)
{
for (int j = 0; j < imageHeight; j++)
{
int tmpColorValue = pixelArray[i, j];
MyColor tmp = new MyColor(tmpColorValue);
if (colors.Contains(tmp))
{
MyColor tmpColor = (from p in colors
where p.colorValue == tmpColorValue
select p).First();
tmpColor.pointList.Add(new MyPoint(i, j));
}
else
{
tmp.pointList.Add(new MyPoint(i, j));
colors.Add(tmp);
}
}
}
}
}
public class MyColor : IEquatable<MyColor>
{
public int colorValue { get; set; }
public List<MyPoint> pointList = new List<MyPoint>();
public MyColor(int _colorValue)
{
colorValue = _colorValue;
}
public bool Equals(MyColor other)
{
if (this.colorValue == other.colorValue)
{
return true;
}
return false;
}
}
public class MyPoint
{
public int xCoord { get; set; }
public int yCoord { get; set; }
public MyPoint(int _xCoord, int _yCoord)
{
xCoord = _xCoord;
yCoord = _yCoord;
}
}

If you're getting a stack overflow I would guess that you're not excluding already-checked pixels. The first check on visiting a square should be whether you've been here before.
Also, I was working on a related problem not too long ago and I came up with a different approach that uses a lot less memory:
A queue:
AddPointToQueue(x, y);
repeat
x, y = HeadItem;
AddMaybe(x - 1, y); x + 1, y; x, y - 1; x, y + 1;
until QueueIsEmpty;
AddMaybe(x, y):
if Visited[x, y] return;
Visited[x, y] = true;
AddPointToQueue(x, y);
The point of this approach is that you end up with your queue basically holding a line wrapped around the mapped area. This limits memory usage better than a stack can.
If relevant it also can be trivially modified to yield the travel distance to any square.

Try using AForge.net. I would go for Filter by colors, Threshold and then you could do some Morphology to decrement the black/White zones to lose contact between the objects. Then you could go for the Blobs.